Hi,

i'm currently trying to parse the data from the tn_tweets table for some additional statistics. I have noticed the extra-column, which seems to contain a serialized object/array/hash/whatever.
I am trying to unserialize it with several ruby gems, and they all fail. So i would like to ask: Is this the pure serialized data, or did you modify the serializing algorithm or whatever?

The setup page checks for $_POST["redirect"] to see if you've clicked the "sign in with Twitter" button. However, Firefox (because the Twitter button is an image) supplies redirect_x and redirect_y $_POST variables but not $_POST, meaning that the request isn't detected as having had the Twitter button pressed. This works in Chrome.

I thought that it might be an issue with twitter, but it doesn't look like it.
Doing a search through the db shows that the twitter timestamps and tweetid are exactly the same for all duplicates.
According to the loadtweets.php script I had only 751 tweets, but looking at the front page of my tweetnest install shows 1,498 tweets. Something isn't right.
How can I provide some debugging info?
I'm more than willing to drop the tables and start over a few times.

Hi guys!

From 28th of last month, till today, I’m getting this error when trying to get the new tweets. I thought it was a temporary error, but not…it continues. What is going wrong?
The problem is when trying to connect to Twitter, but why?

Thanks!!

it seems like the setting 'twitter_screenname' isuseless. when you run maintenance/loaduser.php it takes the owner of the OAUTH and puts it in the tn_tweetusers table (that's always the only user there, is that normal?) and that user's tweets are what gets pulled later.

Tested: I edited the tweetusers line and gave it the userID I meant to be archived, from that moment on, maintenance/loadtweets.php started pulling the correct tweets.

got an easier fix?

Firstly, Tweet Nest is really cool :) thanks for the time spent on this. One of the issues I'm having is that on my shared hosting package, the API limit is frequently reached (I'm guessing quite a few other users on the same server use unauthenticated twitter feeds) which means Tweet Nest sometimes struggles to get new tweets.

The only way to really solve this is to implement OAuth, but I've read through the comments on issue #1 and know that it would be a bit of a development and would also require each user of Tweet Nest to register a Twitter app – not ideal. One solution to this would be to make connecting via OAuth an option in the config rather than required. Aside from helping out those of us on shared hosting, OAuth would also allow some quite cool extensions of the app in the future to grab much more data from Twitter.

I'm just throwing this out there as a potential feature, I appreciate your time's precious! If I get some free time I may even implement this in my own fork to test it out.

Thanks again

If I'm installing into a dedicated table, I'd rather not use prefixes, but the installer requires them.

Got this message when setting up Tweet Nest:

Your MySQL client version is too old. Tweet Nest requires MySQL version 4.1 or higher to function. Your client currently has mysqlnd 5.0.7-dev - 091210 - $Revision: 300533 $.

From here:

We are changing our star icon for favorites to a heart and we’ll be calling them likes.

Favorites should no longer be called that, they should be called 'likes'.

Installed Tweetnest successfully:

http://kerim.oxus.net/tweetnest/

But my favorites haven't loaded, and although it says "all tweets ... ever" it only loaded tweets for this year. Otherwise, very nice!

If you use, e.g., the "Unicode Text Converter" (see here) in order to emphasise (parts of) your tweet, chances are that tweetnest will only show question marks for the respective pasted characters.

While arguably the use of special/non-standard characters should be minimised to begin with, "masking" them renders most of the tweets in question illegible.

In maintenance/loadtweets.php#L154 you reset the favorite-flag of all tweets to 0.
I understand that there is a problem handling revoked favorites but reseting alls favorite-markers will lead to trouble if:

• you have multiple users only the favorites of the last user are stored at all
• you have more favorites than twitter wants to show you (and these scripts are meant as a work-around to just this kind of limitation, right?)

If you stored favorites like suggested in #2 you still would be able to revoke (at least some) favorites if you still fetch all favorites you are offered, otherwise I am unsure whether conserving as many favorites as possible is more important or allowing revocations.
Allowing multi-user-favorites seems to be impossible without storing any information about the individual favorites.

Keep the good work up!

I thought that it might be an issue with twitter, but it doesn't look like it.
Doing a search through the db shows that the twitter timestamps and tweetid are exactly the same for all duplicates.
According to the loadtweets.php script I had only 751 tweets, but looking at the front page of my tweetnest install shows 1,498 tweets. Something isn't right.
How can I provide some debugging info?
I'm more than willing to drop the tables and start over a few times.

I am running Ubuntu Serverx64 10.04.2, mysql 5.1.41, php5 5.3.2

(I'm sorry I can't say more about the problem I've got, but I don't really know what's going on.)

Every time I try click the 'sign in with twitter' button (after the other fields have been filled in of course), I get the message "You must authorize Tweet Nest to use your Twitter account before continuing." But I've already done that (and Tweet Nest appears in the list of authorized applications in Twitter).

Any ideas on what I should be looking for to fix this?

Thanks!

Hi
Just installed Tweetnest on a shared Hoster, whit PHP 5.5.3 in FPM/FastCGI-Mode.
calling "/maintenance/loadtweets.php" won't work. It seems as if the "$_SERVER['PHP_AUTH_USER']" is not set.
I read some threads on stackoverflow, but the mentioned workarounds won't fix my problem.

My solution is removing the Password in the config File, comment out line 47 in mpreheader.php and securing the "maintenance"-Folder with .htaccess and .htpasswd

Has anyone the same problems and perhaps a fix that works?

best regards
Heiko

I'm intermittently seeing this error when the loadtweets.php cronjob runs. I haven't received this error ever when running the job manually.

"PHP Fatal error:  Cannot use object of type stdClass as array in loadtweets.php on line 171"

When running Tweet Nest on PHP 5.5, it prints out warnings regarding the preg_replace() calls on line 390, 391 and 392 inside tweetnest/inc/html.php, as they make use of the "/e" flag. That flag is deprecated as of PHP 5.5.0, as stated in the change log for the function, and should be replaced with preg_replace_callback().

I haven't looked into how to remove the warning, but I might do it in a couple of days if I get the time.

I had TN working fine 18 months ago then let it slide. So I got it working again. I managed to imprt the archive and resolve the dulicate twwet issue. However shortly afterwards it stopped working. I've only just got around to trying to solve this. The cron response give me:-

X-Powered-By: PHP/5.3.28
Content-type: text/plain

Trying to grab from user_id=xxxxxxxxx...
Importing:
User ID: xxxxxxxxx
Total tweets: 3200, Approx. page total: 16 Newest tweet I've got: 582298538580795393 Retrieving page #1:
92 new tweets on this page
 - 631084877979471873 Tue Aug 11 12:48:54 +0000 2015
Snip
 - 583227038464634880 Wed Apr 01 11:18:56 +0000 2015 Retrieving page #2:
0 new tweets on this page
All tweets collected. Reconnecting to DB...
Inserting into DB...
Done!

Syncing favourites...
Retrieving page #1:
96 total favorite tweets on this page
 - 99431148320591873 Fri Aug 05 10:46:47 +0000 2011
snip
 - 24095652130 Fri Sep 10 11:16:33 +0000 2010
21 favorite own tweets so far
Retrieving page #2:
0 total favorite tweets on this page
21 favorite own tweets so far
Updated favorites!Trying to grab from user_id=0...
Importing:
User ID: 0
Retrieving page #1:
1 new tweets on this page
A Twitter API error occured: Sorry, that page does not exist.

I would be grateful for some pointers on how to resolve this.

Charlie

(I didn't found a discussion regarding this enhancement in previous issues)

I literally can't find anyway to install this app without changing my mysql password, since it seems to require one when I in fact have none.

I was recently writing a personal Wordpress widget to display my most recent tweets. The widget makes a direct call to the tweetnest database tables. When I output the tweet data on my page, many special characters such as umlauts or emoji came out garbled (see attachment). It took me quite a while until I figured out, that if I convert the tweet from Windows-1252 encoding to UTF-8, the characters show up correctly.

Is that expected behavior? Shouldn't the tweet text that gets saved be the actual unicode characters, especially since the MySQL field is marked utf8_general_ci?

Is it possible this has something to do with my individual server configuration (regular Apache, etc.), or is this something that goes wrong when Tweetnest saves the tweets to the database.

I realize if this gets fixed, the database entries all need to be converted, or past installations become incompatible with newer versions.
Maybe it could be a config option, defaulting to the old Windows-1252 format, but with the option to declare UTF-8 on installation or first launch. That way, we'd have the choice to manually convert the database and switch to an actual unicode format.

Hello!
Understanding that Tweetnest scans my Twitter Favourites ("Likes") for only my own tweets that I have Liked…
Twitter Rate Limits to 15 requests per 15 minutes on Favorites/Likes
https://dev.twitter.com/rest/public/rate-limits

The situation
At some point back in June, my cronjob failed for a few weeks. When I fixed it up again, there were more than 15 pages (woah) of Favorites for tweetnest to scan.
Because Tweetnest dieout() when the rate limit message is hit, it never completes its process of checking my Likes, pulling out those that are of my own tweets, saving them to the database, and resetting whatever marker that keeps it going back again next time.

End effect is I do not have self-liked tweets in Tweetnest updated anymore, and running loadtweets.php fails with "Rate Limit Exceeded" for Likes every time.

Imagined Fix
When Rate Limit is hit, instead of dieout() (line 215), catch the error, stop the requests, but process what has been retrieved and increment counter so that when the script is run 20 minutes later, it does continues where it left off.

I'm not sure my skills in PHP are good enough to do this, but I might try sometime if I can find time. Figured I'd post the issue here though. :)

Tweet entities, which allow the highlighting of certain link-able parts of a tweet in HTML, have indices that specify where in the original tweet these transformations are to be made (character indexes).

However, Tweet Nest does not apply them until after some characters are transformed into HTML entities (such as ">" to ">"), and in some cases SmartyPants stupefication which can turn an em dash into three hyphens. In both these cases, more characters are created than were originally in the tweet, and thus the tweet entity indices get faulty. Faulty tweet entity indices result in misplaced links that cover part of the tweet text that should not have been linked, and thus renders some tweets unable to be read properly because this text is cut off.

This issue is complicated because it only occurs some of the time, usually on old tweets loaded recently, where the list of entities does not cover everything, for historical reasons. Old tweets that were loaded before the occurrence of tweet entities have no entities at all. Old tweets that were loaded after the occurence of tweet entities seem to have at least hashtag entities, regardless of the tweet creation date, and maybe more. Additionally, some tweets appear HTML entity-decoded to begin with (no raw ">" characters), while some tweets don't, depending on client and other things.

All these things result in inconsistent displaying behaviour depending on tweet source, tweet posting time, and the time it was loaded into the system, and can thus result in a poor viewing experience in Tweet Nest.

I don't have an immediate solution, but maybe some clever heads here on GitHub can look at the issue. Otherwise, we might have to look into using an external PHP library that can properly parse these tweet entities while simultaneously encoding HTML entities correctly.

For relevant code, see the tweetHTML, linkifyTweet and entitifyTweet functions in inc/html.php.

There will be changes to the Twitter-API which will affect Tweetnest as e.g. the page= parameter is to be removed.

Full story at https://dev.twitter.com/blog/api-housekeeping

The month list on the side stopped loading for me about two months ago: http://kiriska.com/tweety/ I was on the last release of TweetNest, beta 1.1, I guess? loadtweets.php had also stopped working.

I just download the master copy from here and reinstalled/reauth'd everything. I was able to use loadtweets.php and loaduser.php fine now, but the months still don't display. Navigating to month/day pages also doesn't work: http://kiriska.com/tweety/2015/11 All the tweets are still safe in the database, so I'm not sure what the issue is.

I have a few hundred favorite tweets on my account, but I can’t get Tweetnest to pull them. I seem to recall that they appeared previously, but now they’re gone.

I’ve tried “loadtweets” and got this:

––––-
Trying to grab from user_id=############…
Importing:
User ID: ############
Total tweets: , Pages: 0
Newest tweet I’ve got: 344311076940292096
Nothing to insert.

Syncing favourites…
Updated favorites!Trying to grab from user_id=0…
Importing:
User ID: 0
Total tweets: , Pages: 0
Nothing to insert.

Syncing favourites…
Updated favorites!

RSS was always broken on api.twitter and now they shut it down entirely. are you planning on offering RSS output from TN?

I generally favorite other people's tweets (starring your own is a little self-congratulatory), and I'd love a setting so Tweetnest can download every tweet I've favorited.

Calling loadtweets.php in the browser works.
loadlog.txt shows attempt:
Attempted load Mon, 06 Apr 2015 16:09:47 +0200

But a crontab like the following doesn't work:
50 1 * * * php /server path//tweetnest/maintenance/loadtweets.php
No entry in loadlog.txt

Any idea? Is my crontab syntax wrong?

My Tweetnest (and many others by googling a bit) is throwing a PHP error on preg_replace():

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in /mysitepath.com/tweetnest/inc/html.php on line 390

Seems like this is deprecated as of PHP 5.5 and my host (Dreamhost) upgraded sometime in the recent past.

:390

$html = preg_replace("/$lookbehind\b(((https?:\/\/)|www\.).+?)(([!?,.\"\)]+)?(\s|$))/e", "_linkifyTweet_link('$1', '$2', '$3', '$4')", $str);

Also happens on 392 and 393:
:392

$html = preg_replace("/\B\@([a-zA-Z0-9_]{1,20}(\/\w+)?)/e", "_linkifyTweet_at('$1', '$2')", $html);

:393

$html = preg_replace("/\B\#([\pL|0-9|_]+)/eu", "_linkifyTweet_hashtag('$1', '$2')", $html);

Can anyone advise on how to modify using the preg_replace_callback() syntax?

I confirmed with another buddy that this was happening to him as well. Anyone else?

Pretty much, it's not even September (at the time of this writing), and I have 2 calendar-months created: September and October 2011. So for example, my latest tweet of the night:

"10:58 PM, Aug 31st, 2011 via bitly"

... is under October 2011 grouping.

I hope that is somewhat clear. Please advise when you get a chance. Thanks in advance.

Regards.

Tweets have not been loading the last few days. At first I thought it was an API issue (with Twitter shutting down the 1.0 API) so I changed all references to the API in loadtweets.php to 1.1 but that didn't fix things. The current error I'm getting when I run loadtweets is as follows (with the id changed):

Retrieving page #1: 1/statuses/user_timeline.json?user_id=XXXXXXXX&include_rts=true&include_entities=true&count=200&since_id=344449689871081472
1 new tweets on this page

Warning: get_object_vars() expects parameter 1 to be object, array given in /home/golddave/public_html/tweetnest/inc/class.twitterapi.php on line 103

Warning: Invalid argument supplied for foreach() in /home/golddave/public_html/tweetnest/inc/class.twitterapi.php on line 103

The same issue occurs when loadtweets is referring to either the 1.0 or 1.1 version of te Twitter API.

It'd be nice to be able to get notifications of new releases by email.

I noticed that the ID from the tweet which it is “in reply to” is stored wrong in the database.
For example the ID from tweet ID 390970989301231616 is stored 3.9097098930123162E+17.

This API change has messed up my archive. I am yet to upgrade because my old archive now has some 10000+ odd duplicate tweets. Some tweets ended up getting posted even 5 times. Example - http://tweets.nspeaks.com/2013/05/31 This started sometime in april. I should have stopped syncing but I detected it late. How to do away with all these duplicate tweets before I upgrade the archive?

on every page, php 5.3.8

Just want to say that this is a fantastic little app you've got going on here.

One problem exists though, and that's in relation to private twitter accounts.

When running the "loadtweets.php" page I get hit with the response:
"Not authorized"

Thoughts on how to integrate the Twitter API into this?

I start using Tweetnest with the latest Version 1.1 Beta.
The crawler fetches all my latest tweets and retweets, but no favorites altough I have many of them.

Could you implement the fetching of the favorites again? :)

Hi there. After my hosting provider upgraded servers to PHP 5.5 I'm seeing a deprecated warning on my Tweetnest page:

Deprecated: preg_replace(): The /e modifier is deprecated, use preg_replace_callback instead in {domain.name}/tweets/inc/html.php on line 390

Is there a way to patch Tweetnest to make this warning to disappear?

I installed tweetnest at this subdomain
https://tweets.lischka.cc/
Imported archives, gave / as the directory, and now I get 404's with
https://tweets.lischka.cc/2014/04
and the like. I cannot figure out the reason. Any idea?

The default .htaccess rules don't work on 1and1 for some reason. They have a weird httpd.conf that seems to destroy rewrite rules. Currently loading anything that needs to be rewritten causes a 404. I have tried tinkering, but everything I do causes a 500 error. I already searched Google to no avail, and tried case-and-pasting from the WP .htaccess, which works, but it didn't help.

.htaccess is way deeper magic than I am used to; I'm out of my league. Suggestions or fixes?

Of all of my hundreds of favorites, Tweetnest is only pulling in one.

Here's the info from loadtweets.php

1 favorite own tweets on this page
Retrieving page \#5: 1/favorites.json?user_id=1013051&count=200&page=4
200 total favorite tweets on this page

1 favorite own tweets on this page
Retrieving page \#6: 1/favorites.json?user_id=1013051&count=200&page=5
199 total favorite tweets on this page

1 favorite own tweets on this page
Retrieving page \#7: 1/favorites.json?user_id=1013051&count=200&page=6
113 total favorite tweets on this page

1 favorite own tweets on this page
Updated favorites!

Any ideas why it only populates one?

I get an exception when ever I set $config['db']['password'] to an empty string. It's not null and it is set, and perfectly valid, yet some check is refusing to let the app run.

If an Emoji is used in a tweet and it is is the leading character, then the whole field will be empty. Tweetnest will then show an empty tweet.

If Emojis are placed somewhere else, they are removed and the text is cropped from this point on.

Line 52 of tweets.js contains:

if($.browser.msie && $.browser.version < 8){

This causes a JS error: "Uncaught TypeError: Cannot read property 'msie' of undefined". This is because jQuery deprecated the feature in 1.3 and since 1.9 it has been removed entirely.

If it were up to me I'd drop support for IE < 8 so the entire shim can be removed; if you insist on supporting older browsers you need to find a better way (modernizr might be a good option, 'future proof' but, imho, way overkill) to detect browser version/features.

As Title.

in twitter.com , i post tweets like:

"测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试"

in tweetnest , it can't show the same tweet which i post in twitter.com:

“测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测试测”

Hi,

the current version generates the right link to pic.twitter Thumbnails, but the links already in the DB don't get fixed.

I used this query to repair it (assuming it's always 50 chars, as it was in my DB):

update `tn_tweets` set `extra` = replace(`extra`,'50:"http://p.twimg','52:"http://pbs.twimg');

This could be placed in upgrade.php.

I'm getting an error saying:

"Your PHP version doesn’t seem to support JSON decoding. This functionality is required to retrieve tweets and is included in the core of PHP 5.2 and above."

I'm on Joyent's servers which have PHP Version 5.2.12. Here is my phpinfo:

http://www.oxus.net/tweetnest/phpinfo.php

Here's my naive translation of the .htaccess RewriteRules you include. There might be a more nginx-y way of getting the same effect, but this appears to work.

server {
    listen *:80;
    server_name tweets.mikedoherty.ca;

    root /var/www/tweets.mikedoherty.ca/tweetnest;
    index index.php;

    access_log /var/log/nginx/tweets.mikedoherty.ca.access.log;
    error_log  /var/log/nginx/tweets.mikedoherty.ca.error.log;

    location /sort {
        rewrite ^/sort /sort.php last;
    }
    location /favourites {
        rewrite ^/favourites /favorites.php last;
    }
    location /favorites {
        rewrite ^/favorites /favorites.php last;
    }
    location /search {
        rewrite ^/search /search.php last;
    }
    if ($uri ~ ^/([0-9]+)/([0-9]+)/?$) {
        rewrite ^/([0-9]+)/([0-9]+)/?$ /month.php?y=$1&m=$2 last;
    }
    if ($uri ~ ^/([0-9]+)/([0-9]+)/([0-9]+)/?$) {
        rewrite ^/([0-9]+)/([0-9]+)/([0-9]+)/?$ /day.php?y=$1&m=$2&d=$3 last;
    }
    location / {
        try_files $uri $uri/ /index.php;
    }
    location ~ \.php$ {
        include fastcgi_params;
        fastcgi_intercept_errors on;
        fastcgi_pass php-fcgi;
    }
    include w3ctc.conf;
}