URL harvesting script
Project ID: 1228257772
Project Details
- Status: Closed (Cancelled)
- Posted: 12/2/2008 at 17:42 EST
- Cancelled: 12/24/2008 at 0:34 EST
- Project Creator:
- Budget: $200
- Description: I need a script or desktop application (for windows vista) to harvest website addresses (URLs) for me. I will enter a keyword phrase like "cheap hosting" and up to three directory URLs (eg. http://dir.yahoo.com/Business_and_Economy/Business_to_Business/Communications_and_Networking/Internet_and_World_Wide_Web/Network_Service_Providers/Hosting/Website_Hosting/) for each keyword phrase.
For each keyword phrase, you will then need to get website addresses from the following sources for me (using "cheap hosting" as an example) -
1) http://www.google.com/sponsoredlinks?q=cheap+hosting&btnG=Search+Sponsored+Links - Up to the first 100 results
2) Sponsored results that appear for http://search.yahoo.com/search?p=cheap+hosting - Up to the first 100 results
3) First 100 Google search results for "cheap hosting"
4) First 100 Yahoo results for "cheap hosting"
5) The first 100 results available from each directory (up to three directories per keyword phrase) specified.
6) The first 100 results from Google Maps
7) The first 100 results from http://www.google.com/search?q=allinurl%3Acheap+hosting - filtered to show only the URLs that actually have one or more of the keywords in the domain name itself. (So cheapcar.com and fasthosting.com is ok, but not computers.com/cheaphosting.htm)
I will need the ALL results in the form of website.com (NOT website.com/djdjd/ururu.htm)in a csv text file, separated into the five categories above.
I will do about 10 keyword phrases at a time.
The script must have a setting to specify a random delay in seconds between searches to avoid being blocked by Google.
- Tags:
| Project Bids |
| Programmer | Bid |
Delivery Time | Time of Bid | Rating | |
| intelliglobalservice | $100 | 2 days | 12/3/2008 at 4:30 EST | (No Feedback Yet) | |
| Hello. Very interesting project for us. Please check PMB for details. | |||||
| snixins | $100 | 2 days | 12/6/2008 at 2:29 EST | (No Feedback Yet) | |
| Can do it friend ... | |||||
| rana32 | $150 | 5 days | 12/12/2008 at 1:04 EST | (21 reviews) |
|
| Hi, can be done, please read my PM thanks | |||||
| mrgooddeal | $180 | 3 days | 12/3/2008 at 2:59 EST | (97 reviews) |
|
| -Time & +Quality | |||||
| valkhorn |
$199 | 5 days | 12/2/2008 at 23:23 EST | (20 reviews) |
|
| As described in the PMB. | |||||
| rsdsoft |
$200 | 5 days | 12/2/2008 at 21:21 EST | (177 reviews) |
|
| I specialize in search engines parsing and data scrapping. Please check PMB. | |||||
| darrenclark | $200 | 10 days | 12/2/2008 at 18:02 EST | (14 reviews) |
|
| If it is okay with you that it is a desktop application running using the .NET runtime, then I can get this project done for you. | |||||
| aruhat | $300 | 10 days | 12/13/2008 at 4:37 EST | (7 reviews) |
|
| Hi Please see PMB. Regards Pratik | |||||
| creatorulcom | $550 | 4 days | 12/2/2008 at 17:49 EST | (83 reviews) |
|
| Professional solutions | |||||






