Differences

This shows you the differences between two versions of the page.

--- spark:windows_dev_environment [2017/11/27 13:35] – [Install/Configure WinUtils] david
+++ spark:windows_dev_environment [2025/03/27 15:42] (current) – external edit 127.0.0.1
@@ Line 3: / Line 3: @@
 Inspiration stolen/taken from: [[https://jaceklaskowski.gitbooks.io/mastering-apache-spark/spark-tips-and-tricks-running-spark-windows.html|Running Spark Applications on Windows - Jacek Laskowski]]
+====== Download ======
 Download:
   * Spark: [[https://spark.apache.org/downloads.html]]
@@ Line 45: / Line 46: @@
 ===== Run it =====
 <code>
-C:\APPLICATIONS\Spark\spark-2.2.0-bin-hadoop2.7\bin>spark-shell2.cmd
+cd C:\APPLICATIONS\Spark\spark-2.2.0-bin-hadoop2.7\bin
+spark-shell2.cmd
+</code>
+Custom run... (local master, 2 cores)
+<code>
+spark-shell2.cmd --master local[2]
+</code>
+Output:
+<code>
 Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
 Setting default log level to "WARN".
@@ Line 68: / Line 79: @@
 Type :help for more information.
 </code>
+Test command:
+<code>
+spark.range(1).withColumn("status", lit("All seems fine. Congratulations!")).show(false)
+</code>
+exit:
+<code>
+sys.exit
+</code>
+----
+  * Web Interface: [[http://localhost:4040/jobs/]]
+    * Configuration: [[http://localhost:4040/environment/]]