At the start of 2018, I began retweeting all the most valuable information I found on twitter, as part of a learning process. Then, I would periodically scroll back and manually paste the links into the channels of a discord server, organized by category.
The problem with that, is if you are scrolling through your history, it has to load, and it’s easy to miss that particualr tweet where I know I left off.
Once I gave up scrolling my twitter history manually, I used a share button integration with discord, to directly send a given link to the proper channel.
Now I’m taking a break from twitter, transitioning from discord to spreadsheets, and in the process of consolidating a few years of link collecting on a variety of subjects. I have a lot to organize between 4 twitter accounts, a dozen discord servers where I organized links on different subjects, and a variety of other sources….
Export Tweets to CSV
So, this is the hard part. Finding a free app that will get as many tweets as you need.
For now, I know vicinitas.io allows you to download up to 3200 tweets to csv, which was enough for me.
In the future, this page will fill with everything related to exporting and publishing content from twitter.
From there, I threw the CSV in Libre Office Calc (an ok free version of Excell). I was able to filter the 3200 tweets to include only those related to identity.
I am going to share more about how, exactly, I make the content from csv. In the meantime:
This is a game changer! While searching for resources to add in this post, I found rob-murray/jekyll-twitter-plugin!!!
- twitterdev/tweet-updates - This repository contains information about the updates to Tweet formats for attachments and simplified replies. -developer.twitter.com - tweet updates
Twitter Related Gists
- yanofsky/tweet_dumper.py - A script to download all of a user’s tweets into a csv
- arikfr/export_lists.py - Simple script to export Twitter lists to CSV]
- vickyqian/twitter crawler.txt - A Python script to download all the tweets of a hashtag into a csv
Twitter GitHub Repos
- kennethreitz/twitter-scraper - Scrape the Twitter Frontend API without authentication.
- mkearney/rtweet - bird R client for interacting with Twitter’s [stream and REST] APIs
- hrbrmstr/21-recipes - An R/rtweet edition of Matthew A. Russell’s Python Twitter Recipes Book
- twintproject/twint - An advanced Twitter scraping & OSINT tool written in Python that doesn’t use Twitter’s API, allowing you to scrape a user’s followers, following, Tweets and more while evading most API limitations.
- taspinar/twitterscraper - Scrape Twitter for Tweets
- AdrienGuille/TweetStreamer - A command line tool for collecting tweets via Twitter’s public streaming API
- DocNow/twarc - A command line tool (and Python library) for archiving Twitter JSON
- Jefferson-Henrique/GetOldTweets-python - A project written in Python to get old tweets, it bypass some limitations of Twitter Official API.
- jennybc/scream - Get replies and quotes of a tweet
- snovvcrash/tweetlord - bird Twitter profile dumper (downloader) with authorization swapping
- Datamine/Archive-Tweets - Archive and Delete Liked and Posted Tweets
- renatolond/mastodon-twitter-poster - Crossposter to post statuses between Mastodon and Twitter
Bots - Code
- freeCodeCamp/100DaysOfCode-twitter-bot - Twitter bot for #100DaysOfCode @_100DaysOfCode
- bonzanini/Book-SocialMediaMiningPython - Companion code for the book “Mastering Social Media Mining with Python
- botwiki/botwiki.org - Tutorials, articles, datasets and other resources for creating useful, interesting, artistic and friendly online bots.
- huginn/huginn - Create agents that monitor and act on your behalf. Your agents are standing by!
- MikeMcQuaid/TwitterDelete - Delete your old, unpopular tweets.
- vzqzac/twgitbot - A node.js bot that checks a github repo changes and tweets it to your Twitter account
- Kikobeats/fetch-timeline-cli - Fetch Twitter user’s timeline from your terminal zap.
- tylerpearson/twitter-most-followed-scripts - Scripts to find the most commonly followed Twitter accounts by a group of people tmf.tylerp.me
- justinlittman/twitter-scraper - A tool for scraping tweet ids from the Twitter website.
- geofurb/Ornitholog - Open-source Twitter collection and archiving tool for tracking specific topics and collecting bulk data.
- motazsaad/tweets-collector - Collect tweets (tweets corpus) using Twitter API. Collection can be based on hashtags, keywords, geographical location
- HoussemDellai/social-media-templates - Provides templates for using Twitter, Youtube, Instagram, Flickr, Foursquare and Eventbrite API with Xamarin Forms.