29

I am running into an issue with my Maven POM file where its unable to find spark dependency and is returning with error: Failed to read artifact descriptor for org.apache.spark:spark-streaming-kafka_2.10:jar:1.2.1

I've confirmed its not an issue with any corporate firewall as every other dependency is loaded in properly, just this one.

I have also been able to confirm in my maven settings it is attempting to pull from the following repo. I tried deleting the .m2 repo on my local machine in order to reload it, still no dice.

http://repo.maven.apache.org/maven2/org/apache/spark/spark-streaming-kafka_2.10/1.2.1/

Attached below is my pom file

<groupId>my.group.id</groupId>
<artifactId>sentiment</artifactId>
<version>1.0-SNAPSHOT</version>
<name>NPITWITTER</name>

<properties>
</properties>

    <dependencies>
        <dependency>
            <groupId>com.sparkjava</groupId>
            <artifactId>spark-core</artifactId>
            <version>1.1.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-streaming-kafka_2.10</artifactId>
            <version>1.2.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_2.10</artifactId>
            <version>1.2.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-streaming_2.10</artifactId>
            <version>1.2.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-hive_2.10</artifactId>
            <version>1.2.1</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-sql_2.10</artifactId>
            <version>1.2.1</version>
        </dependency>


    </dependencies>

user2100493
  • 1,218
  • 4
  • 15
  • 25
  • 1
    This dependency correctly exists: http://mvnrepository.com/artifact/org.apache.spark/spark-streaming-kafka_2.10/1.2.1 so the problem is elsewhere. – Tunaki Jan 30 '16 at 18:54

8 Answers8

116

In Intellij, I had the same issue,

I did the proxy settings also, and ran the command mvn –U clean install but it's not working.

I got its resolved with the below setting enabled (highlighted in yellow)

In Intellij, go to File -> Settings -> Build, Execution, Deployments -> Maven Check the box, ‘Always update snapshots

Intellij Settings window snapshot

YCF_L
  • 51,266
  • 13
  • 85
  • 129
Raju
  • 1,169
  • 2
  • 7
  • 3
16

Resolved it, was due to a proxy setting that was enabled. Re-ran with mvn -U clean install and all was repaired.

einverne
  • 6,116
  • 5
  • 41
  • 89
user2100493
  • 1,218
  • 4
  • 15
  • 25
6

Following worked for IDEA 17,

  • Go to File
  • Select Settings
  • Select Build, Execution, Deployments
  • Select Build Tools from drop down
  • Select Maven from drop down
  • Tick the Always update snapshots check box
prime
  • 12,918
  • 12
  • 87
  • 123
6

I guess IntelliJ comes with and ambedded Maven installation, so just make sure your IDE is pointing to the right Maven home directory: and User settings file: values, these are under:

  • Go to File
  • Select Settings
  • Select Build, Execution, Deployments
  • Select Build Tools from drop down
  • Select Maven from drop down
Sandoval0992
  • 1,257
  • 2
  • 16
  • 23
2

I had just solved the issue. Please, go in the Maven tab of the Build, Execution, Deployments and set it to the settings provided.

enter image description here

Arefe
  • 9,121
  • 13
  • 85
  • 143
0

In my case, it was proxy related problem. I am highlighting few of areas where you may need to verify and correct the same.

  1. Intellij proxy settings from preference window
  2. maven settings.xml entry from your ~/.m2/settings.xml or /usr/local/Cellar/maven//libexec/conf/settings.xml
  3. System proxy

Hope this will help.

Dhrubo
  • 655
  • 1
  • 11
  • 32
0

The above solution had to extend further to resolve my issue(In IntelliJ idea).

Step 1

Go to File -> Settings -> (Build, Execution, Deployment) -> Maven and tick the Always update snapshot option and then click on Apply -> Ok

enter image description here

Step 2

Go to File -> Invalidate Cache/Restart -> Invalidate and Restart

enter image description here

Step 3

Build the project after step 2 and see whether the error is fixed. If not, Restart the computer and then run the project.

I had to follow all the 3 steps to get the error fixed

Niroshan Ratnayake
  • 2,499
  • 2
  • 13
  • 17
-1

Go to file => Settings => Build, Execution, Deployment => Maven => Maven home directory: change from Bundled (Maven 3) to Bundled (Maven 2)