Website scraping using jquery and ajax
I want to be able to manipulate the html of a given url. Something like html scraping. I know this can be done using curl or some scraping library.But i would like to know if it is possible to use jquery to make a get request to the url using ajax and retrieve the html of the url, and run jquery code on the html returned ?
Thank You
Answer
I would like to point out that there are situations where it is perfectly acceptable to use jQuery to scrape screens across domains. Windows Sidebar gadgets run in a 'Local Machine Zone' that allows cross domain scripting.
And jQuery does have the ability to apply selectors to retreived html content. You just need to add the selector to a load() method's url parameter after a space.
The example gadget code below checks this page every hour and reports the total number of page views.
<html>
<head>
<script type="text/javascript" src="jquery.min.js"></script>
<style>
body {
height: 120px;
width: 130px;
background-color: white;
};
</style>
</head>
<body>
Question Viewed:
<div id="data"></div>
<script type="text/javascript">
var url = "http://stackoverflow.com/questions/1936495/website-scraping-using-jquery-and-ajax"
updateGadget();
inervalID = setInterval("updateGadget();", 60 * 1000);
function updateGadget(){
$(document).ready(function(){
$("#data").load(url + " .label-value:contains('times')");
});
}
</script>
</body>
</html>