1

I am new on Apache Spark. I am trying to setup Apache Spark to my Macbook. I download file "spark-2.4.0-bin-hadoop2.7" from Apache Spark official web site.
When I try to run ./bin/spark-shell or ./bin/pyspark I get Permission denied error.
I want to just run spark on my local machine.
I also tried to give permission to all folders but it does not help. Why do I this error?

Taher A. Ghaleb
  • 4,810
  • 5
  • 27
  • 41
stef
  • 71
  • 2
  • 10
  • Have you tried `chmod +x` – Geno Chen Dec 02 '18 at 12:13
  • After chmod +x spark-2.4.0-bin-hadoop2.7 I take this for pyspark => /Users/apple/spark-2.4.0-bin-hadoop2.7/bin/spark-submit: line 27: /Users/apple/spark-2.4.0-bin-hadoop2.7/bin/spark-class: Permission denied /Users/apple/spark-2.4.0-bin-hadoop2.7/bin/spark-submit: line 27: exec: /Users/apple/spark-2.4.0-bin-hadoop2.7/bin/spark-class: cannot execute: Undefined error: 0 – stef Dec 02 '18 at 12:16

2 Answers2

2

This should solve your problem chmod +x /Users/apple/spark-2.4.0-bin-hadoop2.7/bin/*

Then you could try executing bin/pyspark (spark shell in python) or bin/spark-shell (spark shell in scala).

Oli
  • 7,903
  • 3
  • 20
  • 42
  • I tried but i take still take same errors for spark-shell and pyspark=> chmod +x /Users/apple/spark-2.4.0-bin-hadoop2.7/bin/* -bash: ./bin/spark-shell: /usr/bin/env: bad interpreter: Operation not permitted apple-MacBook-Air:spark-2.4.0-bin-hadoop2.7 apple$ ./bin/pyspark /Users/apple/spark-2.4.0-bin-hadoop2.7/bin/spark-submit: /Users/apple/spark-2.4.0-bin-hadoop2.7/bin/spark-class: /usr/bin/env: bad interpreter: Operation not permitted /Users/apple/spark-2.4.0-bin-hadoop2.7/bin/spark-submit: line 27: /Users/apple/spark-2.4.0-bin-hadoop2.7/bin/spark-class: Undefined error: 0 – stef Dec 02 '18 at 13:06
  • Have you opened the files with an editor? In that case, have a look at https://stackoverflow.com/questions/9952612/mac-os-usr-bin-env-bad-interpreter-operation-not-permitted – Oli Dec 02 '18 at 13:22
1

I solve this issue by adding /libexec folder to spark home path

set $SPARK_HOME to

/usr/local/Cellar/apache-spark/<your_spark_version>/libexec

stef
  • 71
  • 2
  • 10