Subscribe to DSC Newsletter
Pandiyarajan
  • Male
  • Chennai,TamilNadu
  • India
Share on Facebook
Share

Gifts Received

Gift

Pandiyarajan has not received any gifts yet

Give a Gift

 

Pandiyarajan's Page

Profile Information

Company:
Sciera Solution Pvt Ltd.
Seniority:
Technical
Industry:
Big Data
Short Bio:
Working in Sciera,As Perl Developer.
Interests:
Finding a new position, Networking, New venture

Pandiyarajan's Blog

Need solution To Crawl 200 million Web Page in 10-1 month Using Perl and hadoop is this possible?

Posted on November 21, 2013 at 10:10pm 1 Comment

Hi all Greetings from Prajan,

 I am Prajan(Pandiyarajan), Working As a Perl Developer in Sciera Solution.I have 2+ year in perl and Big Data .

 I got a difficult task from my manager ,that was i need to Crawl 200 million URL in same Domain with in 20-1 month time ,i have tried in Perl i got Maximum 250 hits only in 60 seconds.I heard will make this using Hadoop But i don't know hadoop ,Any one can You give a solution for my task with detail then how can i reach…

Continue

Comment Wall

You need to be a member of Data Science Central to add comments!

Join Data Science Central

  • No comments yet!
 
 
 

Videos

  • Add Videos
  • View All

© 2020   TechTarget, Inc.   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service

console.log("HostName");