How to fix NullPointerException, jsoup element

Question

I was using Java Eclipse and imported jsoup package, in order to scrape from the Amazon website. The program simply goes to a search page, see the number of results and if any changes make notifications on the left side of the screen and constantly reload pages with a timer. Well it was working well and after a while, it shows me an error constantly. Sometimes it executes properly.

I have tried to use if(select=!null), but it shows me an empty result. This means that selector sometimes scrapes data sometimes gets nulls.

    package main;

       import java.awt.AWTException;
       import java.awt.SystemTray;
       import java.io.IOException;
       import java.net.MalformedURLException;

       import org.jsoup.Jsoup;
       import org.jsoup.nodes.Document;
       import org.jsoup.nodes.Element;
       import org.jsoup.select.Elements;
       import java.util.Timer; 
       import java.util.TimerTask;
       class amazon1 extends TimerTask{
               public static int result1;
               public static int result2;
               public static int result3;
         public void run() {



        try {
            final Document doc = 
        Jsoup.connect("http://www.amazon.com/s?k=hello&i=stripbooks-intl- 
        ship&ref=nb_sb_noss")
                    .userAgent("Mozilla/17.0")
                    .get()

                    ;
      Elements select = doc.select("div.a-section.a-spacing- 
              small.a-spacing-top-small>span");
            Element first = select.first();
            String contentText = first.text();
            amazon1.result1 = 
             Integer.parseInt(contentText.replaceAll("[\\D]",""));
                } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }


        //



        if(amazon1.result1>amazon1.result2) {
        System.out.println(amazon1.result1);
        amazon1.result2 = amazon1.result1;
        amazon1.result3 = amazon1.result1 - amazon1.result2;
           if (SystemTray.isSupported()) {
                DisplayTrayIcon td = new DisplayTrayIcon();
                try {
                    td.displayTray();
                } catch (MalformedURLException | AWTException e) {
                    // TODO Auto-generated catch block
                    e.printStackTrace();
                }
            } else {
                System.err.println("System tray not supported!");
            }

        } else {
         ;
        }






      }

   }



     public class Main {
       public static void main(String[] args) throws IOException {

         Timer timer = new Timer(); 
            TimerTask task = new amazon1(); 
            timer.schedule(task, 3000, 5000);



     }





   }

Error is..

    Exception in thread "Timer-0" java.lang.NullPointerException

  at main.amazon1.run(Main.java:31)

  at java.util.TimerThread.mainLoop(Timer.java:555)

  at java.util.TimerThread.run(Timer.java:505)

NullPointer exception comes from selecting class element. The output should be 11620000 and notification from the right side of the screen.

Answer 1

Samuel Philipp is not wrong suggesting that this question refers to the classic "What is a NullPointerExcelption" Question here on Stackoverflow (What is a NullPointerException, and how do I fix it?)

However, since you experience the NullPointerException only sometimes, it is worthwhile to think about the reasons why this is happening.

My guess would be the following. Sometimes, the website you are trying to reach is not available, or it is blocking your request, if you repeat it too often for example. In that case the line

final Document doc = Jsoup.connect(
        "http://www.amazon.com/s?k=hello&i=stripbooksintl-ship&ref=nb_sb_noss")
    .userAgent("Mozilla/17.0")
    .get();

will end up with doc == null. This is why the select method then fails with the Exception.

To fix this, you either terminate the catch (IOException e) block, or you check for doc being null after the connect method.

} catch (IOException e) {
    // it seems the website is not reachable or the content is not according to the expectations.
    System.err.println("website not reachable or content malformed");
    // you may need to set the result1, result2, result3 variables accordingly here
    amazon1.rsult1 = 0;
    amazon1.rsult2 = 0;
    amazon1.rsult3 = 0;
    return;
}