1

I am trying to make use of MultipleTextOutputFormat to write the data in customized way, here is the example that I am referring to : https://stackoverflow.com/a/26051042/2485454

However, I am getting an error after executing above code :

java.lang.RuntimeException: java.lang.NoSuchMethodException : $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$RDDMultipleTextOutpu‌​tFormat.()

koiralo
  • 21,620
  • 4
  • 46
  • 70
aks
  • 979
  • 1
  • 7
  • 16

1 Answers1

0

It's not really possible diagnose this fully without a project, but when you run into NoSuchMethodExceptions, you can be pretty sure the issue is a compile-time vs. run-time dependency conflict. Perhaps you are using a library that's pulling in a mismatched spark or hadoop library. Or perhaps you're just compiling against a different spark version than you are running on.

Try removing all your extra libraries and/or running the code in a pure spark REPL and build it back up from there.

easel
  • 3,912
  • 24
  • 28