Ruby Mechanize: Follow a Link

themirror picture themirror · Jul 14, 2011 · Viewed 9.1k times · Source

In Mechanize on Ruby, I have to assign a new variable to every new page I come to. For example:

  page2 = page1.link_with(:text => "Continue").click
  page3 = page2.link_with(:text => "About").click
  ...etc

Is there a way to run Mechanize without a variable holding every page state? like

  my_only_page.link_with(:text => "Continue").click!
  my_only_page.link_with(:text => "About").click!

Answer

Niels Kristian picture Niels Kristian · Jul 19, 2011

I don't know if I understand your question correctly, but if it's a matter of looping through a lot of pages dynamically and process them, you could do it like this:

    require 'mechanize'

    url = "http://example.com"
    agent = Mechanize.new
    page = agent.get(url) #Get the starting page

    loop do
      # What you want to do on the page - ex. extract something...
      item = page.parser.css('.some_item').text
      item.save

      if link = page.link_with(:text => "Continue") # As long as there is still a nextpage link...
        page = link.click
      else # If no link left, then break out of loop
        break
      end
    end