[TwitterBridge] Meteor does not update Twitter Bridge Import posts

Submitted by Jordan Conway on Mon, 09/27/2010 - 11:57

Issue ID:	2773
Issue Category:	enhancement
Component:	plugin
Priority:	normal
Status:	active
Assigned:	Unassigned
Version:	0.9
Keywords:	twitter

Realtime updates to not include posts from the Twitter Bridge import, you must refresh the page to get the Twitter posts.
This was originally not done due to private tweets, but now that we do not import private tweets this functionality could be included.

Updates

#1

Submitted by Brion Vibber on Tue, 09/28/2010 - 19:30.

Probably we can redo a bunch of the code in twitterstatusfetcher to more cleanly fit with other parts of the system...

Currently saveStatus() is skipping the standard Notice::saveNew() and manually doing an insert followed by the saveStatusMentions() method which pulls up the mentions and such, then after the saveStatus() call we manually call the inbox delivery. Eek!

What we probably should be doing instead is going via Notice::saveNew() and passing in the mentioned users etc. We've got infrastructure now for overriding the various default processing, and the distribution code checks the source so gatewayed posts will go out where they should (inboxes, mail notifications, jabber, realtime) and not where they shouldn't (outgoing syndication etc).

Hopefully. :D

#2

Submitted by Brion Vibber on Tue, 11/30/2010 - 18:11.

Title:

Meteor does not update Twitter Bridge Import posts

» [TwitterBridge] Meteor does not update Twitter Bridge Import posts

Tweaking title to indicate that the guilty party here is TwitterBridge, not Realtime or Meteor.

#3

Submitted by 0xAFFE on Thu, 03/17/2011 - 13:47.

Keywords:

+twitter

I have looked at the code and it seems to me that the current code in twitterimport.php is using saveNew, but the meteor plugin still does not push messages to the timeline.

Whats the issue here?

#4

Submitted by Brion Vibber on Mon, 03/21/2011 - 12:59.

Current code only uses Notice::saveNew for repeats, not for regular posts. For those, TwitterImport does a manual insert to the notice table, and does partial inbox distribution handling itself, which is all fairly icky.

Main difficulty with fully merging these may depend on how much we care about accuracy of delivery.

Normally, dealing with an incoming message looks something like this:
* something comes in (local post, or from another service) from a given profile
* from the list of @-mentions etc in the content, we add some explicit delivery
* from our server's own knowledge of who our users follow, we add implicit delivery from following
* Notice::saveNew() stuffs the notice into the notice table, and a bunch of delivery info, and triggers email notifications and any outgoing magic we have

But when doing Twitter import, it looks like this:
* we go through the list of bridged users
* for each user, we request a list of new statuses that they have received
* for each one, we may or may not need to import it to a local notice
* once that's done, we add the (newly or previously) imported notice to that user's inbox
* ...continue until done with this user's inbox.
* ...continue until done with all bridged users.

So the regular StatusNet processing path deals with each notice once and pumps out to all users, while the Twitter import processing path deal with each user in turn, and may deal with each status/notice multiple times, each time only knowing one receiving user.

If we switch to using the realtime-delivery Site Streams API in future, this actually doesn't change much -- it still has the same properties that we're basically seeing each user's individual timeline as a separate entity, and have to process each status multiple times each in the context of a single receiver.

Now, if we give up on tracking Twitter's actual destination deliveries, but just work from the friends lists and explicit @-mentions, we could probably switch that around -- so the first time we received each Twitter status, we can just do a single import, deliver it to anyone @-mentioned or that we know is a follower of the poster. The regular code paths could be used for notification etc, and we'd just discard the status the next time we see it on another timeline, knowing that we've already processed it.

But, that relies on having complete and up to date friends lists, and assumes that nothing else will trigger statuses going into your Twitter timeline that we can't see.