Explore topic-wise InterviewSolutions in Current Affairs.

Java applications are not stored in Apache, it can be only connected to a other Java webapp hosting webserver using the mod_jk connector. mod_jk is a replacement to the elderly mod_jserv. It is a completely new TOMCAT-Apache plug-in that handles the communication between Tomcat and Apache.

Several reasons:

• mod_jserv was too complex : Because it was ported from Apache/JServ, it brought with it lots of JServ specific bits that aren’t needed by Apache.
• mod_jserv supported only Apache : Tomcat supports many WEB servers through a compatibility layer named the jk library. SUPPORTING two different MODES of work became problematic in terms of support, DOCUMENTATION and bug fixes. mod_jk should fix that.
• The layered approach : provided by the jk library makes it easier to support both Apache1.3.x and Apache2.xx.
• Better support for SSL : mod_jserv couldn’t reliably identify whether a request was made via HTTP or HTTPS. mod_jk can, using the newer Ajpv13 protocol.

Java applications are not stored in Apache, it can be only connected to a other Java webapp hosting webserver using the mod_jk connector. mod_jk is a replacement to the elderly mod_jserv. It is a completely new Tomcat-Apache plug-in that handles the communication between Tomcat and Apache.

Several reasons:

It DEFINES whether Apache should spawn itself as a CHILD PROCESS (standalone) or keep everything in a SINGLE process (inetd). Keeping it inetd conserves resources.

The ServerType directive is included in Apache 1.3 for BACKGROUND compatibility with older UNIX-based version of Apache. By default, Apache is set to standalone server which means Apache will run as a separate application on the server. The ServerType directive isn’t available in Apache 2.0.

It defines whether Apache should spawn itself as a child process (standalone) or keep everything in a single process (inetd). Keeping it inetd conserves resources.

The ServerType directive is included in Apache 1.3 for background compatibility with older UNIX-based version of Apache. By default, Apache is set to standalone server which means Apache will run as a separate application on the server. The ServerType directive isn’t available in Apache 2.0.

Apache Kafka : It is a distributed and robust messaging system that can handle huge amount of data and allows passage of messages from ONE end-point to another. Kafka is designed to allow a single cluster to serve as the CENTRAL data backbone for a large organization. It can be elastically and transparently expanded without downtime. Data streams are partitioned and spread over a cluster of machines to allow data streams larger than the capability of any single machine and to allow clusters of COORDINATED CONSUMERS.

Whereas.

Apache Storm : It is a real time message processing system, and you can edit or manipulate data in real-time. Storm pulls the data from Kafka and applies some required manipulation. It makes it easy to reliably process unbounded streams of data, doing real-time processing what Hadoop did for batch processing. Storm is simple, can be used with any programming language, and is a lot of fun to use.

Apache Kafka : It is a distributed and robust messaging system that can handle huge amount of data and allows passage of messages from one end-point to another. Kafka is designed to allow a single cluster to serve as the central data backbone for a large organization. It can be elastically and transparently expanded without downtime. Data streams are partitioned and spread over a cluster of machines to allow data streams larger than the capability of any single machine and to allow clusters of coordinated consumers.

Whereas.

Apache Storm : It is a real time message processing system, and you can edit or manipulate data in real-time. Storm pulls the data from Kafka and applies some required manipulation. It makes it easy to reliably process unbounded streams of data, doing real-time processing what Hadoop did for batch processing. Storm is simple, can be used with any programming language, and is a lot of fun to use.

It is the maximum amount of time allotted to the topology to FULLY process a message released by a spout. If the message in not ACKNOWLEDGED in given time FRAME, APACHE Storm will FAIL the message on the spout.

It is the maximum amount of time allotted to the topology to fully process a message released by a spout. If the message in not acknowledged in given time frame, Apache Storm will fail the message on the spout.

Apache Web Server package does not include ASP support. However, a number of projects provide ASP or ASP-like functionality for Apache.

Some of these are:

Apache:ASP :- Apache ASP provides Active Server Pages port to the Apache Web Server with Perl scripting only, and enables developing of dynamic web applications with session management and embedded Perl code. There are also many powerful extensions, including XML taglibs, XSLT rendering, and new events not originally part of the ASP AP.

mod_mono :- It is an Apache 2.0/2.2/2.4.3 module that provides ASP.NET support for the web’s favorite server, Apache. It is hosted inside Apache. Depending on your configuration, the Apache box could be one or a dozen of separate PROCESSES, all of these processes will send their ASP.NET requests to the mod-MONO-server process. The mod-mono-server process in turn can host multiple independent applications. It does this by using Application DOMAINS to ISOLATE the applications from each other, while using a single Mono virtual machine.

Apache Web Server package does not include ASP support. However, a number of projects provide ASP or ASP-like functionality for Apache.

Some of these are:

Apache:ASP :- Apache ASP provides Active Server Pages port to the Apache Web Server with Perl scripting only, and enables developing of dynamic web applications with session management and embedded Perl code. There are also many powerful extensions, including XML taglibs, XSLT rendering, and new events not originally part of the ASP AP.

mod_mono :- It is an Apache 2.0/2.2/2.4.3 module that provides ASP.NET support for the web’s favorite server, Apache. It is hosted inside Apache. Depending on your configuration, the Apache box could be one or a dozen of separate processes, all of these processes will send their ASP.NET requests to the mod-mono-server process. The mod-mono-server process in turn can host multiple independent applications. It does this by using Application Domains to isolate the applications from each other, while using a single Mono virtual machine.

In financial services, Storm can be helpful in preventing

Securities fraud :

1. Perform real-time anomaly detection on known patterns of activities and use learned patterns from prior modeling and simulations.
2. Correlate transaction data with other streams (chat, email, etc.) in a cost-effective parallel processing environment.
3. Reduce query time from hours to minutes on large volumes of data.
4. Build a single platform for operational APPLICATIONS and analytics that reduces total cost of ownership (TCO)

Order routing : Order routing is the process by which an order goes from the end user to an exchange. An order MAY go directly to the exchange from the customer, or it may go first to a broker who then ROUTES the order to the exchange.

Pricing : Pricing is the process whereby a business sets the price at which it will sell its products and services, and may be part of the business’s marketing plan.

Compliance Violations : compliance means conforming to a rule, such as a specification, POLICY, standard or law. Regulatory compliance describes the goal that organizations aspire to achieve in their efforts to ensure that they are aware of and take steps to comply with relevant laws and regulations. And any disturbance in REGARDING compliance is violations in compliance.

In financial services, Storm can be helpful in preventing

Securities fraud :

Order routing : Order routing is the process by which an order goes from the end user to an exchange. An order may go directly to the exchange from the customer, or it may go first to a broker who then routes the order to the exchange.

Pricing : Pricing is the process whereby a business sets the price at which it will sell its products and services, and may be part of the business’s marketing plan.

Compliance Violations : compliance means conforming to a rule, such as a specification, policy, standard or law. Regulatory compliance describes the goal that organizations aspire to achieve in their efforts to ensure that they are aware of and take steps to comply with relevant laws and regulations. And any disturbance in regarding compliance is violations in compliance.

To read from the log FILES, you can configure your spout and emit PER line as it read the log. The OUTPUT then can be assign to a bolt for ANALYZING.

To read from the log files, you can configure your spout and emit per line as it read the log. The output then can be assign to a bolt for analyzing.

YES, Apache CONTAINS a SEARCH engine. You can search a report NAME in Apache by using the “Search title”.

Yes, Apache contains a Search engine. You can search a report name in Apache by using the “Search title”.

MultiViews search is enabled by the MultiViews Options. It is the GENERAL name given to the Apache SERVER’s ability to provide language-specific document variants in response to a request. This is documented quite thoroughly in the CONTENT negotiation description page. In addition, Apache Week carried an article on this subject entitled It then chooses the BEST MATCH to the client’s requirements, and returns that document.

MultiViews search is enabled by the MultiViews Options. It is the general name given to the Apache server’s ability to provide language-specific document variants in response to a request. This is documented quite thoroughly in the content negotiation description page. In addition, Apache Week carried an article on this subject entitled It then chooses the best match to the client’s requirements, and returns that document.

No. Root process OPENS port 80, but never LISTENS to it, so no user will actually enter the site with root RIGHTS. If you kill the root process, you will see the other roots disappear as well.

No. Root process opens port 80, but never listens to it, so no user will actually enter the site with root rights. If you kill the root process, you will see the other roots disappear as well.

Field grouping in storm uses a mod HASH function to decide which task to SEND a TUPLE, ensuring which task will be processed in the correct order. For that, you don’t require any cache. So, there is no time-out or limit to KNOWN field values.

The stream is partitioned by the fields specified in the grouping. For EXAMPLE, if the stream is grouped by the “user-id” field, tuples with the same “user-id” will always go to the same task, but tuples with different “user-id”‘s may go to different tasks.

Field grouping in storm uses a mod hash function to decide which task to send a tuple, ensuring which task will be processed in the correct order. For that, you don’t require any cache. So, there is no time-out or limit to known field values.

The stream is partitioned by the fields specified in the grouping. For example, if the stream is grouped by the “user-id” field, tuples with the same “user-id” will always go to the same task, but tuples with different “user-id”‘s may go to different tasks.

We can check syntax for httpd CONFIGURATION file by using following COMMAND.

httpd –S

This command will dump out a description of how Apache PARSED the configuration file. Careful examination of the IP addresses and server names may help UNCOVER configuration mistakes.

We can check syntax for httpd configuration file by using following command.

httpd –S

This command will dump out a description of how Apache parsed the configuration file. Careful examination of the IP addresses and server names may help uncover configuration mistakes.

The first two are REMNANTS from the NCSA times, and generally you should be FINE if you delete the first two, and STICK with httpd.conf.

srm.conf :- This is the DEFAULT FILE for the ResourceConfig directive in httpd.conf. It is processed after httpd.conf but before access.conf.

access.conf :- This is the default file for the AccessConfig directive in httpd.conf.It is processed after httpd.conf and srm.conf.

httpd.conf :-The httpd.conf file is well-commented and mostly self-explanatory.

The first two are remnants from the NCSA times, and generally you should be fine if you delete the first two, and stick with httpd.conf.

srm.conf :- This is the default file for the ResourceConfig directive in httpd.conf. It is processed after httpd.conf but before access.conf.

access.conf :- This is the default file for the AccessConfig directive in httpd.conf.It is processed after httpd.conf and srm.conf.

httpd.conf :-The httpd.conf file is well-commented and mostly self-explanatory.

Apache is a WEB (HTTP) server, not an APPLICATION server. The BASE package does not include any such functionality. PHP PROJECT and the mod_perl project allow you to work with databases from within the Apache environment.

Apache is a Web (HTTP) server, not an application server. The base package does not include any such functionality. PHP project and the mod_perl project allow you to work with databases from within the Apache environment.

SSL (Secure SOCKET Layer) data transport requires encryption, and many governments have restrictions upon the import, EXPORT, and use of encryption technology. If Apache included SSL in the base package, its DISTRIBUTION would involve all sorts of legal and BUREAUCRATIC issues, and it would no longer be freely available. Also, some of the technology required to talk to current clients USING SSL is patented by RSA Data Security, who restricts its use without a license.

SSL (Secure Socket Layer) data transport requires encryption, and many governments have restrictions upon the import, export, and use of encryption technology. If Apache included SSL in the base package, its distribution would involve all sorts of legal and bureaucratic issues, and it would no longer be freely available. Also, some of the technology required to talk to current clients using SSL is patented by RSA Data Security, who restricts its use without a license.

Storm UI is used in MONITORING the topology. The Storm UI provides information about errors happening in tasks and fine-grained stats on the THROUGHPUT and latency PERFORMANCE of each COMPONENT of each running topology.

Storm UI is used in monitoring the topology. The Storm UI provides information about errors happening in tasks and fine-grained stats on the throughput and latency performance of each component of each running topology.

There are a variety of configurations you can set PER topology. A list of all the configurations you can set can be found here. The ones prefixed with “TOPOLOGY” can be overridden on a topology-specific basis (the other ones are cluster configurations and cannot be overridden).

Here are some common ones that are set for a topology:

1 Config.TOPOLOGY_WORKERS : This sets the number of worker processes to use to execute the topology. For example, if you set this to 25, there will be 25 Java processes across the cluster executing all the tasks. If you had a combined 150 parallelism across all components in the topology, each worker process will have 6 tasks running within it as threads.

2 Config.TOPOLOGY_ACKER_EXECUTORS : This sets the number of executors that will track tuple trees and detect when a spout tuple has been fully processed By not SETTING this VARIABLE or setting it as null, Storm will set the number of acker executors to be EQUAL to the number of workers configured for this topology. If this variable is set to 0, then Storm will immediately ack tuples as soon as they come off the spout, effectively disabling reliability.

3 Config.TOPOLOGY_MAX_SPOUT_PENDING : This sets the maximum number of spout tuples that can be pending on a single spout task at once (pending means the tuple has not been acked or failed YET). It is highly recommended you set this config to prevent queue explosion.

4 Config.TOPOLOGY_MESSAGE_TIMEOUT_SECS : This is the maximum amount of time a spout tuple has to be fully completed before it is considered failed. This value defaults to 30 seconds, which is sufficient for most topologies.

5 Config.TOPOLOGY_SERIALIZATIONS : You can register more serializers to Storm using this config so that you can use custom types within tuples.

There are a variety of configurations you can set per topology. A list of all the configurations you can set can be found here. The ones prefixed with “TOPOLOGY” can be overridden on a topology-specific basis (the other ones are cluster configurations and cannot be overridden).

Here are some common ones that are set for a topology:

1 Config.TOPOLOGY_WORKERS : This sets the number of worker processes to use to execute the topology. For example, if you set this to 25, there will be 25 Java processes across the cluster executing all the tasks. If you had a combined 150 parallelism across all components in the topology, each worker process will have 6 tasks running within it as threads.

2 Config.TOPOLOGY_ACKER_EXECUTORS : This sets the number of executors that will track tuple trees and detect when a spout tuple has been fully processed By not setting this variable or setting it as null, Storm will set the number of acker executors to be equal to the number of workers configured for this topology. If this variable is set to 0, then Storm will immediately ack tuples as soon as they come off the spout, effectively disabling reliability.

3 Config.TOPOLOGY_MAX_SPOUT_PENDING : This sets the maximum number of spout tuples that can be pending on a single spout task at once (pending means the tuple has not been acked or failed yet). It is highly recommended you set this config to prevent queue explosion.

4 Config.TOPOLOGY_MESSAGE_TIMEOUT_SECS : This is the maximum amount of time a spout tuple has to be fully completed before it is considered failed. This value defaults to 30 seconds, which is sufficient for most topologies.

5 Config.TOPOLOGY_SERIALIZATIONS : You can register more serializers to Storm using this config so that you can use custom types within tuples.

A CombinerAggregator is used to combine a set of tuples into a single field. It has the following signature:

public INTERFACE CombinerAggregator {
T INIT (TridentTuple tuple);
T combine(T val1, T val2);
T zero();
}

Storm calls the init() METHOD with each tuple, and then repeatedly calls the combine()method until the partition is processed. The values passed into the combine() method are partial aggregations, the RESULT of combining the values RETURNED by calls to init().

A CombinerAggregator is used to combine a set of tuples into a single field. It has the following signature:

public interface CombinerAggregator {
T init (TridentTuple tuple);
T combine(T val1, T val2);
T zero();
}

Storm calls the init() method with each tuple, and then repeatedly calls the combine()method until the partition is processed. The values passed into the combine() method are partial aggregations, the result of combining the values returned by calls to init().

A tuple COMING off a SPOUT can trigger thousands of tuples to be created based on it. Consider.

for example:

the streaming word count topology:TopologyBuilder builder = new TopologyBuilder();
builder.setSpout("sentences", new KestrelSpout("kestrel.backtype.com",
22133,
"sentence_queue",
new StringScheme()));
builder.setBolt("split", new SplitSentence(), 10)
.shuffleGrouping("sentences");
builder.setBolt("count", new WORDCOUNT(), 20)
.fieldsGrouping("split", new Fields("word"));

This topology reads sentences off a Kestrel queue, splits the sentences into its constituent words, and then emits for each word the number of times it has seen that word before. A tuple coming off the spout triggers many tuples being created based on it: a tuple for each word in the sentence and a tuple for the updated count for each word.

Storm considers a tuple coming off a spout “fully processed” when the tuple tree has been exhausted and every message in the tree has been processed. A tuple is considered failed when its tree of messages FAILS to be fully processed within a specified timeout. This timeout can be CONFIGURED on a topology-specific basis using the Config.TOPOLOGY_MESSAGE_TIMEOUT_SECS configuration and defaults to 30 seconds.

A tuple coming off a spout can trigger thousands of tuples to be created based on it. Consider.

for example:

the streaming word count topology:TopologyBuilder builder = new TopologyBuilder();
builder.setSpout("sentences", new KestrelSpout("kestrel.backtype.com",
22133,
"sentence_queue",
new StringScheme()));
builder.setBolt("split", new SplitSentence(), 10)
.shuffleGrouping("sentences");
builder.setBolt("count", new WordCount(), 20)
.fieldsGrouping("split", new Fields("word"));

This topology reads sentences off a Kestrel queue, splits the sentences into its constituent words, and then emits for each word the number of times it has seen that word before. A tuple coming off the spout triggers many tuples being created based on it: a tuple for each word in the sentence and a tuple for the updated count for each word.

Storm considers a tuple coming off a spout “fully processed” when the tuple tree has been exhausted and every message in the tree has been processed. A tuple is considered failed when its tree of messages fails to be fully processed within a specified timeout. This timeout can be configured on a topology-specific basis using the Config.TOPOLOGY_MESSAGE_TIMEOUT_SECS configuration and defaults to 30 seconds.

Storm USES Zookeeper for COORDINATING the cluster. Zookeeper is not used for message passing, so the load that Storm places on Zookeeper is quite LOW. Single NODE Zookeeper clusters should be sufficient for most cases, but if you want failover or are deploying large Storm clusters you may want larger Zookeeper clusters. Instructions for deploying Zookeeper are here.

A few notes about Zookeeper deployment :

It’s critical that you run Zookeeper under supervision, since Zookeeper is fail-fast and will exit the process if it encounters any error case. See here for more details.

It’s critical that you set up a cron to compact Zookeeper’s data and transaction logs. The Zookeeper daemon does not do this on its own, and if you don’t set up a cron, Zookeeper will quickly run out of disk space.

Storm uses Zookeeper for coordinating the cluster. Zookeeper is not used for message passing, so the load that Storm places on Zookeeper is quite low. Single node Zookeeper clusters should be sufficient for most cases, but if you want failover or are deploying large Storm clusters you may want larger Zookeeper clusters. Instructions for deploying Zookeeper are here.

A few notes about Zookeeper deployment :

It’s critical that you run Zookeeper under supervision, since Zookeeper is fail-fast and will exit the process if it encounters any error case. See here for more details.

It’s critical that you set up a cron to compact Zookeeper’s data and transaction logs. The Zookeeper daemon does not do this on its own, and if you don’t set up a cron, Zookeeper will quickly run out of disk space.