spark:windows_dev_environment

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
spark:windows_dev_environment [2017/11/27 13:43] – davidspark:windows_dev_environment [2021/01/12 21:08] (current) – external edit 127.0.0.1
Line 3: Line 3:
 Inspiration stolen/taken from: [[https://jaceklaskowski.gitbooks.io/mastering-apache-spark/spark-tips-and-tricks-running-spark-windows.html|Running Spark Applications on Windows - Jacek Laskowski]] Inspiration stolen/taken from: [[https://jaceklaskowski.gitbooks.io/mastering-apache-spark/spark-tips-and-tricks-running-spark-windows.html|Running Spark Applications on Windows - Jacek Laskowski]]
  
 +====== Download ======
 Download: Download:
   * Spark: [[https://spark.apache.org/downloads.html]]   * Spark: [[https://spark.apache.org/downloads.html]]
Line 46: Line 47:
 <code> <code>
 cd C:\APPLICATIONS\Spark\spark-2.2.0-bin-hadoop2.7\bin cd C:\APPLICATIONS\Spark\spark-2.2.0-bin-hadoop2.7\bin
-C:\APPLICATIONS\Spark\spark-2.2.0-bin-hadoop2.7\bin>spark-shell2.cmd+spark-shell2.cmd 
 +</code> 
 + 
 +Custom run... (local master, 2 cores) 
 +<code> 
 +spark-shell2.cmd --master local[2]
 </code> </code>
  
Line 77: Line 83:
 <code> <code>
 spark.range(1).withColumn("status", lit("All seems fine. Congratulations!")).show(false) spark.range(1).withColumn("status", lit("All seems fine. Congratulations!")).show(false)
 +</code>
 +
 +exit:
 +<code>
 +sys.exit
 </code> </code>
  
 ---- ----
   * Web Interface: [[http://localhost:4040/jobs/]]   * Web Interface: [[http://localhost:4040/jobs/]]
 +    * Configuration: [[http://localhost:4040/environment/]]
  • spark/windows_dev_environment.1511790210.txt.gz
  • Last modified: 2021/01/12 21:08
  • (external edit)