php - Big Data with cURL -
i own vps server, want server have big data 15k usernames.
what need, send usernames curl request with: website.com/getaccount.php?username= every minute.
so solution case .. don't want make cron because take 15k jobs , bad.
in language optimize 15k users ? bash ? perl ? php ? , if can make separate files , each file content 5k users.
please give me solution, alot ..
if understand question correctly, want distribute big job using job server such gearman
gearman extremely flexible, , can use batch process large number of jobs in parallel quickly.
when comes language should use...i use whatever you're comfortable in. have processed large jobs ( much larger 1 described ) using bash + php in past. php has wrapper libraries can use things started pretty quickly.
hope helps go in correct direction.
suggested links:
gearman php extension - quick start
Comments
Post a Comment