1.

What Spark-SQL does, how it’s benefits to programmer to interact with database? And Syntax of creating SQL Context?

Answer»

Spark SQL provides programmatic abstraction in the form of data frame and data set which can work the principal of distributed SQL query engine. Spark SQL simplify the INTERACTION to the large amount of data through the dataframe and dataset.

  • Spark-SQL provide a relation processing along with spark functional programming.
  • Support querying data using SQL and HIVE query language.
  • Support Datasource API, Dataframe API, Interpreter & Optimizer, SQL Service.
  • Spark-SQL also providing the new API called Dataset which has CAPABILITY of both Dataframe and core.
  • Spark-SQL I much OPTIMIZE to perform SQL query-based operation on flat file, json.
  • Spark SQL support variety of language LIKE: Java, Scala, Python and R.
  • Code Snippet: val sqlContext = new SQLContext( sc: SparkContext)  
  • Dataframe can be create using below approach: 
  • Structured data files: 
  • Tables in Hive: 
  • External databases:
  • Using existing RDD:  

Spark SQL plays a vital role on optimization technique using Catalyst optimizer, Spark SQL also support UDF, built in function and aggregates function. 



Discussion

No Comment Found