#StackBounty: #scala #apache-spark #compiler-errors WARN ShutdownHookManager: ShutdownHook '$anon$2' timeout,

Bounty: 150

I have spark job which is reading a file, connecting to DB server doing some calculations and then create a different .csv file.

Since yesterday its continue to fail with this error (even tho it creates an output file):

error WARN ShutdownHookManager: ShutdownHook ‘$anon$2’ timeout

How to fix it?

I did search and found that somebody recommend to use ‘explicitly invoke sparkContext.stop() before exiting the application’. but the problem if I am adding this like of code to my spark code I cant compile it, on compilation it throws me an error:

not found: value sparkContext [error] sparkContext.stop() [error]
^ [error] one error found [error] (compile:compileIncremental)
Compilation failed

Get this bounty!!!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.