Ben Sturmfels [Sat, 29 Nov 2014 23:52:11 +0000 (10:52 +1100)]
Use Six's method to determine when running under Python 3.
oauth.py guesses whether it is running under Python 3 by the existance of urllib.parse.urllib_parse and urllib.parse.urlencode. The code then uses urlencode's safe parameter if it believes it's running under Python 3. Unfortunately the guess fails when using Future, a library for writing clean Python 3 code that is backwards compatible with Python 2, since Future manipulates some builtins.
This change applies the version test that isn't confused by the use of Futures. It's the same test used by the Six library.
Mike Verdone [Sat, 24 May 2014 10:41:21 +0000 (12:41 +0200)]
Merge pull request #227 from mponweiser/master
twitter-archiver: Option file path fixed for Windows
Hi, similar to the fix for issue https://github.com/sixohsix/twitter/issues/60, I have adapted the file path for twitter-archiver to work in Windows. Tested under Ubuntu 12.04 and Windows 7, Python 2.7.
Notes:
* 2.6, 2.7, 3.3 pass.
* 3.4 isn't quite yet available, but is coming soon: https://github.com/travis-ci/travis-ci/issues/1989#issuecomment-39315535
* 3.2 fails because it doesn't have u'uncode stuff' but I guess most Py3 users will be ok with 3.3 or 3.4.
* I've not looked into why pypy fails.
* (This PR also contains the same changes as PR #220.)
Mike Verdone [Tue, 15 Apr 2014 14:55:06 +0000 (16:55 +0200)]
Merge pull request #217 from sixohsix/pickled
Support pickling at the risk of breaking some stuff
- support pickling of responses (#82)
- silence Python 2.6 warning (#179)
- this makes rate-limit headers unavailable on boolean or string responses. I'm not sure if that's a problem.
Mike Verdone [Mon, 24 Mar 2014 20:19:47 +0000 (21:19 +0100)]
Merge pull request #215 from wlonk/colorization
Colorization
I've added some logic to align usernames and wrap tweets. It's preliminary, but hopefully useful. I have tied it in to the ANSI format, because that seems the most human-oriented format.
I'm happy to talk about this more and go back and forth on it, and I hope it's useful!
Mike Verdone [Mon, 24 Mar 2014 20:00:27 +0000 (21:00 +0100)]
Merge pull request #216 from edi-bice/master
--follow-redirects hangs indefinitely on certain urls
urllib2 opener open lacked a timeout parameter causing it to hang indefinitely on certain URLs where the host does not send back any data - added timeout=1 for now and will expose as parameter later - also added socket.timeout exception to list of exceptions handled
Edi Bice [Fri, 21 Mar 2014 20:36:58 +0000 (16:36 -0400)]
urllib2 opener open lacked a timeout parameter causing it to hang indefinitely on certain URLs where the host does not send back any data - added timeout=1 for now and will expose as parameter later - also added socket.timeout exception to list of exceptions handled
Mike Verdone [Thu, 20 Mar 2014 17:45:44 +0000 (18:45 +0100)]
Merge pull request #214 from edi-bice/master
Fixed broken rate limiting error handling in archiver/follow.
Twitter changed rate limiting response objects quite a bit in v1.1.
In addition there were bugs due to import of time module and time object from datetime module.
Edi Bice [Thu, 20 Mar 2014 17:21:10 +0000 (13:21 -0400)]
Fixed broken rate limiting error handling in archiver/follow.
Twitter changed rate limiting response objects quite a bit in v1.1.
In addition there were bugs due to import of time module and time object from datetime module.
Mike Verdone [Wed, 12 Mar 2014 20:34:44 +0000 (21:34 +0100)]
Merge pull request #210 from sixohsix/stream_cleanup_two
Stream cleanup twoSecond time... or third time's a charm. I hope this is it.
- code is actually understandable
- NO MORE UNICODE ERRORS
- NO MORE 100% CPU usage
- return data chunks ASAP
- handle "old school" non-blocking mode (for those who still want it)
- works on all stream servers I have access to
Mike Verdone [Sun, 9 Mar 2014 21:11:15 +0000 (22:11 +0100)]
Merge pull request #205 from sixohsix/stream_cleanup
Explicit support for heartbeat handling
Yet another crack at the stream iterator.
Now we explicitly track timeouts and heartbeat timeouts separately. Regular timeouts produce `Timeout` tokens (dicts). Heartbeat timeouts produce `HeartbeatTimeout` and StopIteration if iterated again. This is a solid fix for issue #202.
If the stream is set to use `timeout=None, block=False`, then we yield only data or `None`. If `timeout` is set to a number, we yield data or `Timeout` tokens, never `None`. If we are set to `block=True` then we yield only data.
Also: improve the documentation and remove some code weirdness.
Also: make the example stream program highly configurable to test every bizarre combination from the command line.
Mike Verdone [Mon, 17 Feb 2014 21:29:35 +0000 (22:29 +0100)]
Merge pull request #203 from RouxRC/pr-fix-timeout
Fix streams timeout & hangup behavior + ensure python2.6 compat
Here are the changes to fix the misbehavior of the timeout in case of low tweets by catching Twitter's keep-alive heartbeat signals thanks to the select.select originally added in #178 and still problematic as pointed out by @ksecrist in #202
I also generalized the hangup to all cases since there is no reason to stay in infinite loop after a hangup in non-blocking mode.
And to make things easier and avoid merging issues, I adapted the refacto and fixed python2.6 compatibility from @adonoho's #201
Mike Verdone [Wed, 12 Feb 2014 14:06:23 +0000 (15:06 +0100)]
Merge pull request #199 from adonoho/use-bytearray-buffer
Reduce memory usage by writing directly into byte array buffer.
Gentlefolk,
The whole point of using a `bytearray` as opposed to concatenating reads of `bytes` was to reduce memory usage. This pull request now takes that strategy to its logical conclusion by writing the balance of the chunk directly into the `bytearray`. This saves the creation of a temporary `bytes` array, up to 8KiB in size.
It does this by creating a `memoryview` of the `bytearray`. While this is still an allocation, the `memoryview`s are much smaller and are, presumably, reclaimed faster.
This patch has been running for over 24 hours and has processed over 4 MTw under Python v3.3.3 on OS X 10.8.5. It has also been run for about 10 minutes on Python v2.7.6 on a similar machine. `memoryview` does not appear to have been backported to Python v2.6.*. Hence, this pull request is incompatible with that platform.
Anon,
Andrew
P.S. The changes in this pull request are larger than strictly necessary for adding this functionality. I chose to improve the naming of my variables and move some of them closer to where they are used.
Mike Verdone [Tue, 4 Feb 2014 14:40:12 +0000 (06:40 -0800)]
Merge pull request #197 from RouxRC/pr-rouxrc
Add image support + fix diverses issues
All right, Mike's motivation was contagious and I cherry-picked and merged my pending commits regarding the other parts than the streaming.
This should fix a few more pending issues I tagged back then : #127 #177 #192 #126 #10 #64
I also seized the occasion to add @adonoho among the authors as well.