spark:windows_dev_environment

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
Next revisionBoth sides next revision
spark:windows_dev_environment [2017/11/27 13:35] – [Install/Configure WinUtils] davidspark:windows_dev_environment [2017/11/27 14:53] – david
Line 3: Line 3:
 Inspiration stolen/taken from: [[https://jaceklaskowski.gitbooks.io/mastering-apache-spark/spark-tips-and-tricks-running-spark-windows.html|Running Spark Applications on Windows - Jacek Laskowski]] Inspiration stolen/taken from: [[https://jaceklaskowski.gitbooks.io/mastering-apache-spark/spark-tips-and-tricks-running-spark-windows.html|Running Spark Applications on Windows - Jacek Laskowski]]
  
 +====== Download ======
 Download: Download:
   * Spark: [[https://spark.apache.org/downloads.html]]   * Spark: [[https://spark.apache.org/downloads.html]]
Line 45: Line 46:
 ===== Run it ===== ===== Run it =====
 <code> <code>
-C:\APPLICATIONS\Spark\spark-2.2.0-bin-hadoop2.7\bin>spark-shell2.cmd+cd C:\APPLICATIONS\Spark\spark-2.2.0-bin-hadoop2.7\bin 
 +spark-shell2.cmd 
 +</code> 
 + 
 +Output: 
 +<code>
 Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
 Setting default log level to "WARN". Setting default log level to "WARN".
Line 68: Line 74:
 Type :help for more information. Type :help for more information.
 </code> </code>
 +
 +Test command:
 +<code>
 +spark.range(1).withColumn("status", lit("All seems fine. Congratulations!")).show(false)
 +</code>
 +
 +exit:
 +<code>
 +sys.exit
 +</code>
 +
 +----
 +  * Web Interface: [[http://localhost:4040/jobs/]]
 +    * Configuration: [[http://localhost:4040/environment/]]
  • spark/windows_dev_environment.txt
  • Last modified: 2021/01/12 21:08
  • by 127.0.0.1