Things You Can Do With PHP cURL

Curl is a command line tools to transfer data using various protocol. Curl project was released with libcurl, which is a library for curl that can be use in multi programming language. But it is widely use for PHP with PHP Curl library.

“curl is a command line tool for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and TFTP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, kerberos…), file transfer resume, proxy tunneling and a busload of other useful tricks. ” (Quote from Curl Official site).

cURL, and its PHP extension libcURL, are tools which can be used to simulate a web browser. And there are lots you can do with PHP Curl, here are some of them:

1. Download a file
You can download a file using PHP cURL, and it will download using HTTP request:

set_time_limit(0); // set no time limit to download large file
ini_set('display_errors',true);//Just in case we get some errors, let us know....

$fp = fopen ('path to file', 'w+');//where the file will be saved
$ch = curl_init('http://somedomain.com/filename.zip');//Here is the file we are downloading
curl_setopt($ch, CURLOPT_TIMEOUT, 50);
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_exec($ch);
curl_close($ch);
fclose($fp);

2. Upload with HTTP or FTP protocol
You can upload a file using PHP cURL, and it will upload using HTTP or FTP:
Upload file using HTTP:

$postdata = array();
$postdata ['fieldname'] = "@/path/to/file.zip";  //fieldname should be same as file input box name

$post_url = 'http://somedomain.com/upload.php'; //url to upload file

$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_VERBOSE, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $post_url);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
$response = curl_exec($ch);
curl_close ($ch);

Upload file using FTP:

$filepath = '/path/to/local/file.zip';
$fp = $fp = fopen($filepath, 'r'); //open as read a file

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'ftp://ftp_login:[email protected]/'.'filaname.zip');
curl_setopt($ch, CURLOPT_UPLOAD, 1);
curl_setopt($ch, CURLOPT_INFILE, $fp);
curl_setopt($ch, CURLOPT_INFILESIZE, filesize($filepath));
curl_exec ($ch);
$error_no = curl_errno($ch);
curl_close ($ch);

3. Shorten a URL using Bit.ly
Bit.ly is an URL shortener service, and it allow us to shorten url using their API. And we can connect to their API using PHP Curl:

$longurl='http://www.ivankristianto.com/';
$login='your bit.ly username';
$appkey='your app key';
$api_url = 'http://api.bit.ly/shorten?version=2.0.1&longUrl='.urlencode($longurl).'&format=xml&login='.$login.'&apiKey='.$appkey;
//call the API
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $api_url);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
$response = curl_ex_ec($curl);
curl_close($curl);

//parse the XML response and return the url
$xml_object = new SimpleXMLElement($response);
echo $xml_object->results->nodeKeyVal->shortUrl; //output the shortened url

4. Publish post to your WordPress blog
Wordpress blog have XMLRPC feature to publish a post and publish a comment. We can use cURL to connect to this WordPress blog XMLRPC and publish a post. See my previous post: [HowTo] Publish Post Via XML-RPC In WordPress.

5. Scrape the website content
Basically all the web content is in HTML. You can grap the website content and filter the content and grab the information you want to get. For example you can use cURL to grab the search result from Google, Bing, Yahoo and etc.

function getPage($proxy, $url, $referer, $agent, $header, $timeout) {
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_HEADER, $header);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_PROXY, $proxy);
    curl_setopt($ch, CURLOPT_HTTPPROXYTUNNEL, 1);
    curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
    curl_setopt($ch, CURLOPT_REFERER, $referer);
    curl_setopt($ch, CURLOPT_USERAGENT, $agent);

    $result['EXE'] = curl_exec($ch);
    $result['INF'] = curl_getinfo($ch);
    $result['ERR'] = curl_error($ch);
    curl_close($ch);
    return $result;
}

$result = getPage(
    '[proxy IP]:[port]', // leave it blank of using no proxy
    'url to scrape start with http://',
    'referer url', // leave it blank if if no referer
    'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8',
    1,
    5);

//Process the $result['EXE'] here

6. Update your Facebook status
With cURL we can connect to Facebook and simulate as it is using a web browser and publish a status.

<?php
$status = 'YOUR_STATUS';
$first_name = 'YOUR_FIRST_NAME';
$login_email = 'YOUR_LOGIN_EMAIL';
$login_pass = 'YOUR_PASSWORD';

$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'https://login.facebook.com/login.php?m&amp;next=http%3A%2F%2Fm.facebook.com%2Fhome.php');
curl_setopt($ch, CURLOPT_POSTFIELDS,'email='.urlencode($login_email).'&pass='.urlencode($login_pass).'&login=Login');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_COOKIEJAR, "my_cookies.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "my_cookies.txt");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3");
curl_exec($ch);

curl_setopt($ch, CURLOPT_POST, 0);
curl_setopt($ch, CURLOPT_URL, 'http://m.facebook.com/home.php');
$page = curl_exec($ch);

curl_setopt($ch, CURLOPT_POST, 1);
preg_match('/name="post_form_id" value="(.*)" />'.ucfirst($first_name).'/', $page, $form_id);
curl_setopt($ch, CURLOPT_POSTFIELDS,'post_form_id='.$form_id[1].'&status='.urlencode($status).'&update=Update');
curl_setopt($ch, CURLOPT_URL, 'http://m.facebook.com/home.php');
curl_exec($ch);
?>

7. Test your download speed connection
cURL have information of the connection status. You can grab the information and show you the how long the download time and how fast your download speed.

<?php error_reporting(E_ALL | E_STRICT);

// Initialize cURL with given url
$url = 'http://download.bethere.co.uk/images/61859740_3c0c5dbc30_o.jpg';
$ch = curl_init($url);

curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_USERAGENT, 'Sitepoint Examples (thread 581410; http://www.sitepoint.com/forums/showthread.php?t=581410)');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 2);
curl_setopt($ch, CURLOPT_TIMEOUT, 60);

set_time_limit(65);

$execute = curl_exec($ch);
$info = curl_getinfo($ch);

// Time spent downloading, I think
$time = $info['total_time']
      - $info['namelookup_time']
      - $info['connect_time']
      - $info['pretransfer_time']
      - $info['starttransfer_time']
      - $info['redirect_time'];

// Echo friendly messages
header('Content-Type: text/plain');
printf("Downloaded %d bytes in %0.4f seconds.n", $info['size_download'], $time);
printf("Which is %0.4f mbpsn", $info['size_download'] * 8 / $time / 1024 / 1024);
printf("CURL said %0.4f mbpsn", $info['speed_download'] * 8 / 1024 / 1024);

echo "nncurl_getinfo() said:n", str_repeat('-', 31 + strlen($url)), "n";
foreach ($info as $label => $value)
{
	printf("%-30s %sn", $label, $value);
}

8. Get your adsense earning
Most blogger monetize his website using adsense, including me. You can grab your adsense earning using cURL:
Download the script. (Save it as php file)
Source from: http://planetozh.com/blog/my-projects/track-adsense-earnings-in-rss-feed/

9. Post to twitter
Twitter have their API, and we can use cURL to post our tweet to twitter through their API.

<?php
// Set username and password
$username = 'username';
$password = 'password';
// The message you want to send
$message = 'is twittering from php using curl';
// The twitter API address
$url = 'http://twitter.com/statuses/update.xml';
// Alternative JSON version
// $url = 'http://twitter.com/statuses/update.json';
// Set up and execute the curl process
$curl_handle = curl_init();
curl_setopt($curl_handle, CURLOPT_URL, "$url");
curl_setopt($curl_handle, CURLOPT_CONNECTTIMEOUT, 2);
curl_setopt($curl_handle, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl_handle, CURLOPT_POST, 1);
curl_setopt($curl_handle, CURLOPT_POSTFIELDS, "status=$message");
curl_setopt($curl_handle, CURLOPT_USERPWD, "$username:$password");
$buffer = curl_exec($curl_handle);
curl_close($curl_handle);
// check for success or failure
if (empty($buffer)) {
    echo 'message';
} else {
    echo 'success';
}
?>

10. Download a video from YouTube
With cURL we can scrape the YouTube website to get the video url and download it.
Download the script here.