Caching a Twitter feed, by periodically saving tweets to a local file has a couple of good advantages:
If you’ve followed my previous steps to authenticate a Twitter user timeline and created a JQuery Twitter feed then the following Tweet caching solution only requires three steps:
The PHP script has just a few extra lines – as before, it uses the json_encode function to convert the returned array of tweets to .json format and a couple of other file handling functions to create and write a new file .txt file. The PHP is set to save as .txt instead of .json since the .json file format is not always supported on old windows servers (and you would have to setup a new MIME type through IIS).
<?php session_start(); require_once('twitteroauth/twitteroauth/twitteroauth.php'); //Path to twitteroauth library $twitteruser = "twitterusername"; $notweets = 30; $consumerkey = "12345"; $consumersecret = "123456789"; $accesstoken = "123456789"; $accesstokensecret = "12345"; function getConnectionWithAccessToken($cons_key, $cons_secret, $oauth_token, $oauth_token_secret) { $connection = new TwitterOAuth($cons_key, $cons_secret, $oauth_token, $oauth_token_secret); return $connection; } $connection = getConnectionWithAccessToken($consumerkey, $consumersecret, $accesstoken, $accesstokensecret); $tweets = $connection->get("https://api.twitter.com/1.1/statuses/user_timeline.json?screen_name=".$twitteruser."&count=".$notweets); //Check twitter response for errors. if ( isset( $tweets->errors[0]->code )) { // If errors exist, print the first error for a simple notification. echo "Error encountered: ".$tweets->errors[0]->message." Response code:" .$tweets->errors[0]->code; } else { // No errors exist. Write tweets to json/txt file. $file = $twitteruser."-tweets.txt"; $fh = fopen($file, 'w') or die("can't open file"); fwrite($fh, json_encode($tweets)); fclose($fh); if (file_exists($file)) { echo $file . " successfully written (" .round(filesize($file)/1024)."KB)"; } else { echo "Error encountered. File could not be written."; } } ?>
Test the tweet cache script by running the php (which should display a success message and file size status) and open the .txt file to make sure it has a load of tweets saved to it.
Depending on your server setup and write permissions, you may need to create a new blank username-tweets.txt file (replacing ‘username’ with your twitter username), upload to the same folder as the PHP script and make sure the .txt file has write permissions set.
Most controls panels should let you run scheduled tasks. I’ve setup tweet caching on a number of Windows and Linux server set-ups using Plesk control panel.
Check out either of the two methods below depending on your server:
Scheduling a PHP task in Plesk & Linux
Scheduling a PHP task in Plesk & Windows
If you’ve followed Part 1 of the Twitter feed tutorial, then you can simply replace the line:
$.getJSON('get-tweets1.1.php',
and point to the new .txt file
$.getJSON('username-tweets.txt?'+Math.random(),
Math.random() should ensure browsers won’t try to retrieve the text file from their cache.
Check the cached twitter feed works by refreshing the page a couple of times – hopefully the tweets should now load a lot faster!
Update 1: 28 Jun 13. Added error handling to avoid overwriting of tweets text file if error encountered. Thanks to Rob K for this one
Hi Tom, thanks for the article. I found it helpful when updating a twitter widget after the recent API depreciation.
One suggestion. The method you’ve outlined does not test for potential response errors. If an error did occur, the message would be written to the json cache file – overwriting “good” tweet data and thus breaking the app.
I wrote the code below to test for this scenario. This should keep the twitter feed up even if something goes wrong with twitter’s service.
$tweets = $connection->get(“your_twitter_url”);
//Check twitter response for errors.
if ( isset( $tweets->errors[0]->code )) {
// If errors exist, print the first error for a simple notification.
echo “Error encountered.”
.””.$tweets->errors[0]->message.””
.”Response code: “.$tweets->errors[0]->code.””;
} else {
// No errors exist. Write tweets to json file.
$file = $twitteruser.”-tweets.json”;
file_put_contents($file, json_encode($tweets));
if (file_exists($file)) {
echo $file . ” successfully written (” .round(filesize($file)/1024).”KB)”;
} else {
echo “Error encountered.”
.”File could not be written.”;
}
}
Hello
I have a problem.
I followed your instructions. Works cron, I get the message in my email of the update, but falls twitter screen with the image loading.
I would like to go one by one as in your website.
What could be the problem?
Sorry my english.
Hi Tom,
Thanks for the tutorials, great job!
But I have run into some problems with this Tweet Caching guide. My tweets are stuck on loading.
The only changes I have made to the code from the working custom twitter feed tutorial is to copy the javascript code for get-tweets.php (adding correct twitteruser and keys) and replace “$.getJSON(‘get-tweets1.1.php’,” with “$.getJSON(‘http://www.archetweb.com/archetweb-tweets.txt?’+Math.random(),” in twitterfeed.js
By the way, the get-tweets.php code in the tutorial has an error in it:
$tweets = $connection—>get(“https://api.twitter.com/1.1/statuses/user_timeline.json?screen_name=”.$twitteruser.”&count=”.$notweets);
I have changed this to $tweets = $connection->get(“https://api…… and checked that the script is successfully writing to the file archetweb-tweets.txt.
Would be great if you could take a look at the live feed as I have no idea what the problem is.
Here’s how I do it, this way no cron job needed.
I did not want to do this via cron job so I feel this should work just fine.
We can use my drop in function mod. this also checks the response based on Rob K’s suggestion.
Our response line.
$tweets = $connection—>get(“https://api.twitter.com/1.1/statuses/user_timeline.json?screen_name=”.$twitteruser.”&count=”.$notweets);
Add this directly below our response.
$ctime = ‘3’; //How many hours to cache our tweets.
$cache = ‘cache/tweets56342774.json’; //cache file name.
$my_tweets = t_cache($cache, $tweets, $ctime);
echo json_encode($my_tweets);
Add this at the end of the get-tweets.php file
//Auto cache tweets to avoid limits PHP 5+ …
function t_cache($cache, $tweets, $ctime){
$tdata = json_encode($tweets);
if (!file_exists($cache)) {
if (!file_exists(‘cache’)) {
mkdir(‘cache’);
}
if (!isset($tweets->errors[0]->code)){
file_put_contents($cache, $tdata);
}
$tweets = json_decode(file_get_contents($cache), true);
}else{
$cache_time = time() – filemtime($cache);
if ($cache_time < 3600 * $ctime) {
$tweets = json_decode(file_get_contents($cache), true);
}elseif(!isset($tweets->errors[0]->code)){
file_put_contents($cache, $tdata);
}
}
return $tweets;
}
I did as you advised to get tweets without cron job , but I get output as null !
But echoing $tweets before doing all this is fetching statuses..
Hi,
Thanks for the great scripts. It’s awesome!
Just some questions here:
1) As the the $.getJSON(‘URL/tweets_feed.txt), it won’t work once I have added the ?’+Math.random().
2) I have cross domain issue on IE, have you come across of this? Because of my JQuery and the generated txt is sitting in different servers.
Thanks,
Hi Tom, thanks for being a life saver. Just a quick question… on longer tweets and RTs it can cut off the end off the tweet and have a “…” at the end. Any ideas on how to force it to show it all?
Example on my footer : http://www.squiders.com – the RT on there
HI Kim, you’re welcome.
1) Have you missed the got the comma on the end, i.e. ?’+Math.random(), ?
2). Checkout using JSONP for cross domain requests on IE: http://www.cypressnorth.com/blog/programming/cross-domain-ajax-request-with-json-response-for-iefirefoxchrome-safari-jquery/
Hi Tom,
Any idea how to use the following script in your script?
[script]
if ($.browser.msie && window.XDomainRequest) {
// Use Microsoft XDR
var xdr = new XDomainRequest();
xdr.open(“get”, “someurl”);
xdr.onload = function () {
var JSON = $.parseJSON(xdr.responseText);
if (JSON == null || typeof (JSON) == ‘undefined’)
{
JSON = $.parseJSON(data.firstChild.textContent);
}
processData(JSON);
};
xdr.send();
} else {
$.ajax({
type: ‘GET’,
url: “someurl”,
processData: true,
data: {},
dataType: “json”,
success: function (data) { processData(data); }
});
}
[/script]
Thanks,
Hi Tom!
First, thank you. Your tutorials have been extremely helpful in getting our twitter feeds back up and running after the deprecation of 1.0.
We’re interested in getting our tweets to cache locally, but aren’t sure how to go about scheduling the php to run. We’re working on Macs, so the links you’ve provided with step 2 above don’t apply to our situation. Will the caching still work without step 2? Is there a Mac equivalent of Plesk?
Any guidance would be greatly appreciated!
Thanks,
Emily
Hi Tom
Thanks for doing these tutorials; they have been a life saver.
I’m having trouble getting the cached feeds to work. I’m also using multiple twitter feeds on the same page, with a dynamically generated search term. Do you know of a solution for how I can implement this?
Daniel
Hello,
I work for a web development company who develop in .NET. We rent a Linux server that will create and host all of our cached twitter feeds.
We have an array of our client information needed to authenticate and run this script and we would like to loop through them all and execute this script. However, when we try to do this using the above script our loop will only execute once.
Do you know of a way for us to loop through this script multiple times (we can’t reload the page each time with new information as this is setup as a cron job).
Many thanks.
So a funny thing happened. I had it all working last night. It creates the .txt file and the JS was doing a fine job of getting it displayed how I wanted it. I checked it again today and it was just stuck on the loader. I hadn’t changed anything since getting it running last night.
I had the same loading error on my dev box and on my live server (dev had no cron, live did). After poking around and kind of giving up, I did a test tweet, loaded tweet.php and all the sudden it worked.
What would kill it that adding a new tweet in the stream would fix? Any help would be appreciated. Thanks for the awesomeness you’ve already handed out with these tutorials!
An update. When I posted yesterday, the cron was broken on my live version, so I got that running. I was hoping that would solve the problem but it did not.
This morning I checked the feed and it is back to being stuck on loading. It seem totally tied to the age of the last tweet, which is so strange to me.
Last tweet was 15 hours ago (time of this post) and it’s stuck on loading. Tab over to twitter and tweet something give the cron time to hit it, and it’s working.
I’m all for motivation to tweet more often but that seems heavy handed 🙂
Anyone else seeing this trouble?
Hi Tom,
We’re using your script to create multiple cached Twitter feeds on a single PHP server as we develop using .ASP and .NET. We then tell our jQuery script to request the files stored on this external server.
For all our browsers this works apart from (you guessed it) IE. It appears to be an issue with ourselves making an external request via our jQuery script. We have written another PHP file on the server that retrieves the contents of the text file and sets a header to support cross browser requests and we then request that but still no success in IE.
We’re wondering if you are aware of any issues when making a request for an external file with IE? We have also tried $.ajax.
Hi Tom, this looks great! unfortunately I cannot get it to work? I’ve followed your instructions and don’t seem to be getting anywhere. Keep getting error – or no display at all. It is writing to the txt file fine after testing. Seems the js isn’t playing ball. hmm 🙁
If you could help me out that would be great! Cheers, Jacob
Hi Tom,
I’ve got the get-tweets.php code writing to a .txt file if I manually run the script, from this .txt file the twitter feed updates. However if I try to write a cronjob to do this automatically I cannot get it to work, im running plesk 11, I have a macbook but I think the host is linux?
The current command I am using for my cronjob is ‘php -q httpdocs/get-tweets.php’ and the email notification returns “can’t open file”. I contacted my web host for reasons as to why this may not be working and they suggested that :
“The following code is the correct code to use:
php -q httpdocs/get-tweets.php
The error “can’t open file” is an error generated by the PHP script, we would recommend that you contact the script developer to find out if there are any issues with running the script as a cronjob.”
my domain is http://www.mdwoodman.co.uk
thanks in advance,
Matt
Hi Tom,
The query was escalated to a higher level of tech support and they replied saying:
“The error “can’t open file” suggests that this is potential a file/path problem. I’ve tried adding the absolute path in the get-tweets.php script”
The script now works so it seems that it was an file path issue in the get-tweets.php. Anyway it is now up and working 🙂
Thanks anyway,
Matt
The file path was corrected in get-tweets.php to:
$file = “/var/www/vhosts/mdwoodman.co.uk/httpdocs/”.$twitteruser.”-tweets.txt”; 🙂
Tom,
It’s totally blank.
I’ve checked your troubleshooting tips, theres no HTML outside of the PHP.
I made an APP, got the keys, plugged them in. Changed username, uploaded the file, uploaded the github library. Everything is in a twitter folder on server. I enter address of PHP file. blank. blank. blank.
Hi Tom
I have the script working just fine…except that whenever my page with the script loads the tweets take a second or 2 to fully load. If you then click on a link to another page before the script has fully loaded a pop-up error appears just saying ‘error -‘. This happens quite frequently due to the tweets taking the time to fully load.
Why might this be?
Thanks a lot
– John
I think this can be done easier instead of working with CRON jobs and pointing to the txt file.
Why not just check in the PHP feed page if the txt file exists and if so check for the age of the file. If the file is older e.g. than 10 minutes then generate a new txt file, else just output the contents of the file.
Something like this :
$cachefile = $twitteruser.”-tweets.txt”;
$file_age = (60 * 10); // 10 minutes
if ( (file_exists($cachefile)) && ( (time() – $file_age) > filemtime($cachefile)) || !file_exists($cachefile))
{
// fetch feed & generate a new txt file, same code as in your sample
}
else
{
// txt file exists and is not outdated yet so just read contents from file and output them
$file = file_get_contents($cachefile, true);
echo $file;
}
HI Tom/Reza,
Does this work in the get-tweets.php file? So something like this?
$tweets = $connection->get(“https://api.twitter.com/1.1/statuses/user_timeline.json?screen_name=”.$twitteruser.”&count=”.$notweets);
$cachefile = $twitteruser.”-tweets.txt”;
$file_age = (60 * 10); // 10 minutes
if ( (file_exists($cachefile)) && ( (time() – $file_age) > filemtime($cachefile)) || !file_exists($cachefile))
{
$file = $twitteruser.”-tweets.txt”;
$fh = fopen($file, ‘w’) or die(“can’t open file”);
fwrite($fh, json_encode($tweets));
fclose($fh);
}
else
{
// txt file exists and is not outdated yet so just read contents from file and output them
$file = file_get_contents($cachefile, true);
echo $file;
}
Does anything need to change in the twitter-feed.js? I would like my twitter feed to be updated every 10 mins.
Many thanks
Rob,
This goes in this example in the get-tweets.php file.
The code then looks like this :
$file_age = (60 * 10); // 10 minutes
$cachefile = $twitteruser.”-tweets.txt”;
if ( (file_exists($cachefile)) && ( (time() – $file_age) > filemtime($cachefile)) || !file_exists($cachefile))
{
$connection = getConnectionWithAccessToken($consumerkey, $consumersecret, $accesstoken, $accesstokensecret);
$tweets = $connection->get(“https://api.twitter.com/1.1/statuses/user_timeline.json?screen_name=”.$twitteruser.”&count=”.$notweets);
//Check twitter response for errors.
if ( isset( $tweets->errors[0]->code )) {
echo “Error encountered: “.$tweets->errors[0]->message.” Response code:” .$tweets->errors[0]->code;
} else {
// No errors exist. Write tweets to json/txt file.
$file = $cachefile;
$fh = fopen($file, ‘w’) or die(“can’t open file”);
fwrite($fh, json_encode($tweets));
fclose($fh);
if (file_exists($file)) {
echo json_encode($tweets);
} else {
echo “Error encountered. File could not be written.”;
}
}
}
else
{
$file = file_get_contents($cachefile, true);
echo $file;
}
Great tool! Maybe I’m missing something easy…but in the javascript file where you enter the JSON path for the tweets I’ve used this:
$.getJSON(‘http://www.kimberlypollock.com/KimPollock312-tweets.txt?’+Math.random(),
My widget will only work based on the exact path specified – and the “www” seems mandatory when linking from external sites. But if a user visits my site as kimberlypollock.com in a desktop browser (not using the “www” ) they will see this
Error: Connection problem. Check file path and www vs non-www in getJSON request.
Can i make it so that the tool will load under both http://www.kimberlypollock.com AND kimberlypollock.com? How would I do this?
how to get live or dynamic feeds on twitter ??
if using username-tweets.txt, it mean the feeds is not dynamic.
Hey there Tom.. Great tutorial.. Everything works great! I have one question.. I have set script to cache every 1 min as explained. It returned email saying everything is ok, but it is not updating my mobile app. I have to close out of app and reopen to get new tweets.
Is there something I am missing?
thanks in advance.
One other question.. Would you know how to implement iScroll to “pull to refresh”? Where in the code would I implement this?
Hey Tom, Thanks a lot for these tutorials. They’ve been an amazing help but some how I’ve managed to hit a road block.
I’ve got my PHP working and its saving to my chosen txt file. I’ve altered my JS file to what as far as I’m aware, is the correct code for pulling from that file now. Sadly I’m getting a flicker of my twitter feed and title and then nothing. I’ve debugged it and found that its not filling my variable but is recognising where its meant to be pulling the data from but isn’t?
I’m a bit of a noob in regards to this. The URL is here.
Any ideas?
*Oh, another note thats important. I’m trying to stream by search and not from a specific account. Maybe thats where my issue is stemming from.
Great tutorial just noticed there was an issue in the error catching in twitterfeed.js
} else if (exception === ‘parsererror’) {
error = ‘Requested JSON parse failed.’;
} else if (exception === ‘timeout’) {
error = ‘Time out error.’;
} else if (exception === ‘abort’) {
error = ‘Ajax request aborted.’;
}
exception should be errorThrown
I ended up writing my own update script because I wanted a crontab solution. One of the biggest problems I had with any twitter feed solution was remotely loading profile images from the incredibly slow pbs.twimg.com. The following code encodes the profile images of tweets and retweets into the json file for much faster page load times.
// replace profile images with base64 strings
$data = json_decode(file_get_contents( $feed, false, $context ));
foreach ($data as $tweets) {
if (is_null($tweets->in_reply_to_status_id)) {
if ($tweets->retweeted_status->user->profile_image_url_https) {
$url = $tweets->retweeted_status->user->profile_image_url_https;
$img = ‘data:image/jpeg;base64,’ . base64_encode(file_get_contents($url));
$tweets->retweeted_status->user->profile_image_url_https = $img;
}
elseif ($tweets->user->profile_image_url_https) {
$url = $tweets->user->profile_image_url_https;
$img = ‘data:image/jpeg;base64,’ . base64_encode(file_get_contents($url));
$tweets->user->profile_image_url_https = $img;
}
}
}
// create cache file on server
$json = json_encode($data);
Hi,
I followed your first tutorial and then this but its not working for me.
I have fetch_tweets.php with the following code:
get(“https://api.twitter.com/1.1/statuses/user_timeline.json?include_entities=true&screen_name=”.$twitteruser1.”&count=”.$notweets1);
$tweetfeed1 = json_encode($tweets1);
/*
echo “”;
print_r($tweetfeed1);
echo “”;
*/
//Check twitter response for errors.
if ( isset( $tweets1->errors[0]->code )) {
// If errors exist, print the first error for a simple notification.
echo “Error encountered: “.$tweets1->errors[0]->message.” Response code:” .$tweets1->errors[0]->code;
} else {
// No errors exist. Write tweets to json/txt file.
$file = $twitteruser1.”-tweets.txt”;
$fh = fopen($file, ‘w’) or die(“can’t open file”);
fwrite($fh, json_encode($tweets));
fclose($fh);
if (file_exists($file)) {
echo $file . ” successfully written (” .round(filesize($file)/1024).”KB)”;
} else {
echo “Error encountered. File could not be written.”;
}
}
?>
I am getting an array when I am printing $tweetfeed1.
I have a index.php with the following code:
Fetch Last Tweet For Multiple Users
body {
font-family:Verdana;
font-size:14px;
}
a:link {
color:#0084b4;
}
And the twitterfeed.js file in which I have used the following line:
$.getJSON(‘http://trivone.in/Trivone/last_tweet_cache/vickynikam1609-tweets.txt?’+Math.random(),
Have also created vickynikam1609-tweets.txt which has been given 777 permissions.
However, when I access the index.php file, it doesn’t show up the tweet and I am stuck at the loading icon.
And on accessing the fetch_tweets.php it gives me a message
“vickynikam1609-tweets.txt successfully written (0KB)”
Can you please guide me as to what is that I am doing wrong?
really great tutorial! For my app I need to expand the script in step 1, to get the $twitteruser from a mysql recordset and add a request from Foursquare API too, can you give an example of how can I do it? thanks!
My hosting doesn’t supports file_get_contents . What should I do?
How long does twitter keep around old profile images though? Since that is not cached to server won’t the image be missing between the time you update your profile pic and the server has not done a new cache yet to get the new image url?