Mining, Exploring, Exporting: Creating Content with CSV Tweet Histories

Around a year ago, I began retweeting all the most valuable information I found on twitter. Then, I would periodically scroll back and manually paste the links into the channels of a discord server, organized by category.

The problem with that, is if you are scrolling through your history, it has to load, and it’s easy to miss that particualr tweet where I know I left off.

Eventually, I gave up. More recently, my workflow has become much more direct. When I see a valuable resource (typically via mobile on a smoke break), I click the share button and send it directly to it’s appropriate channel.

I’ve found this to be quite an efficient solution, however… There’s the matter of what about all those links I re-tweeted with intention to categorize, but never got around to.

What do?

Export Tweets to CSV

So, this is the hard part. Finding a free app that will get as many tweets as you need.

For now, I know vicinitas.io allows you to download up to 3200 tweets to csv, which was enough for me.

In the future, this page will fill with everything related to exporting and publishing content from twitter.

From there, I threw the CSV in Libre Office Calc (an ok free version of Excell). I was able to filter the 3200 tweets to include only those related to identity.

I am going to share more about how, exactly, I make the content from csv. In the meantime:

Resources

This is a game changer! While searching for resources to add in this post, I found rob-murray/jekyll-twitter-plugin!!!

Twitter GitHub Repos

Cross-posting

Bots - Code

Clean Feed

Nerd Stuff

Data

Templates

Hugo Starter Kit

Official Resources

[Permalinks]
  # | :year        | the 4-digit year                           |
  # | :month       | the 2-digit month                          |
  # | :monthname   | the name of the month                      |
  # | :day         | the 2-digit day                            |
  # | :weekday     | the 1-digit day of the week (Sunday = 0)   |
  # | :weekdayname | the name of the day of the week            |
  # | :yearday     | the 1- to 3-digit day of the year          |
  # | :section     | the content's section                      |
  # | :sections    | the content's sections hierarchy           |
  # | :title       | the content's title                        |
  # | :slug        | the content's slug (or title, if no slug)  |
  # | :filename    | the content's filename (without extension) |

  # Examples
  posts = "/:filename/"
  # post = "/:year/:month/:title/"
  notes = "/notes/:filename/"

Publishing Websites Via Hugo

Using different versions of Hugo:

  • Netlify Plus Hugo .20 and beyond

    Until yesterday, if you wanted to use a new version of Hugo on Netlify, you had two options. The first one was to wait for us to install it on our build servers and work around name collisions. Although it was not complicated, you can see by reading this blog post, it’s not very sustainable. The second option was to add the version of the Hugo binary you wanted to use to your repository. Since Hugo is a static binary, this is a very convenient solution if you want to manage it yourself.

    Starting today, if you want to use a specific new version of Hugo on Netlify, you only need to set the environment variable HUGO_VERSION with the version number you want to use. If it’s a valid release number, we’ll install it for you and use that version. You don’t have to wait for us, or manage binaries yourself. For example, if you want to use Hugo 0.20 right now, you can go to your site’s settings (Build and Deploy, Build Environment Variables section) and set HUGO_VERSION to 0.20 in your environment.

Basically, if you use netlify it will build with whatever version you tell it to. Otherwise you need to install specific versions locally. You can just drop the binary of the version you need in the root of that projects repository.

Tutorials

  • Hugo Video Turorials

    This course covers the basics of using Hugo - Static Site Generator. Work your way through the videos and we’ll teach you everything you need to know to create a professional and scalable website or blog!

Hugo Shortcodes

Themes

Academic

If you are somewhat familiar with Jekyll already, you may want to jumping right in with Academic Pages. It’s not exactly simple, but its versatile.

I tried github.com/search?q=hugo+theme, and found that Academic has a toooon of stars. More even than the repository for all of hugos themes in one place !!!

There is a one click fork\deploy with GitHub\GitLab Pages and Netlify.

Seriously just click a button, it creates the repo and publishes it for you.

Learn

Another documentation theme, I think a bit simpler than Academic.

All this started a few months ago when I began creating an awesome list on github, then got into publishing via GitHub Pages.

The Markup languages that get transformed into a website.

header image source

Markdown

TOML - YAML - JSON

Just this for now, will get deeper with structured data, later on.

HTML

Color

Unicode

Meta Tags

CSS

Kramdown

Resources for the Command Line, Git, SSH, and BASH.

I use Ubuntu Linux, so all of my info comes from that perspective. That said, most of it translates pretty simply, and much (most?) of this information is platform independent.

Git

Cheat Sheets

Remotes

Writing Commits

Signing Commits

Submodules

Git Workflow

SSH

BASH - Shell Scripting

  • Bash Guide for Beginners

    This is a practical guide which, while not always being too serious, tries to give real-life instead of theoretical examples. I partly wrote it because I don’t get excited with stripped down and over-simplified examples written by people who know what they are talking about, showing some really cool Bash feature so much out of its context that you cannot ever use it in practical circumstances. You can read that sort of stuff after finishing this book, which contains exercises and examples that will help you survive in the real world.

  • https://www.shellscript.sh/
  • http://www.hypexr.org/bash_tutorial.php
  • https://www.bash.academy/

Sources

Header image: IBM PC AT (8Mhz), 2Mb RAM, 20Mb Hard Disk (Formatted), 128Kb EGA, Enhanced Color Display

Migrating from Jekyll HPSTR to Hugo

I’ve tried to learn Hugo a few different ways, and finally found one that works for me, and I hope will work well for anyone.

This guide and the accompanying repository should help anyone switching from either Jekyll to Hugo, or Hugo to Jekyll.

I’ve got two branches of the same exact website except one is run by Jekyll and one is run by Hugo!!

That’s pretty neat, if you ask me!

I also show examples of the differences between the two, where and how I made changes.

Why

  1. mmistakes/hpstr-jekyll-theme is archived, and you can no longer submit issues or pull-requests.
  2. Hugo has a lot of support from the development community, and is growing quickly in features and popularity.
  3. More options.
  4. Learn new skills.
  5. I know of some indieweb themes.
  6. /themes/ directory allows for easy testing and switching between new themes.

HPSTR Jekyll vs Hugo

I’m looking through different releases of Hugo to see which will run HPSTR, since its an old theme and won’t run in the newest version. If you have a newer suggestion, feel free to shout out. I will also find out what the options are.

Releases

It turns out that each version was released at the same time, and that they were built together from the very beginning!

The nice thing about hpstr-hugo-theme is that its not archived, and if you want to open any issues or pull-requests, that can be done.

Interestingly, but nice to see, there are currently 0 open pull request or issues in the themes repository.

Root Directories

Using different versions of Hugo:

  • Netlify Plus Hugo .20 and beyond

    Until yesterday, if you wanted to use a new version of Hugo on Netlify, you had two options. The first one was to wait for us to install it on our build servers and work around name collisions. Although it was not complicated, you can see by reading this blog post, it’s not very sustainable. The second option was to add the version of the Hugo binary you wanted to use to your repository. Since Hugo is a static binary, this is a very convenient solution if you want to manage it yourself.

    Starting today, if you want to use a specific new version of Hugo on Netlify, you only need to set the environment variable HUGO_VERSION with the version number you want to use. If it’s a valid release number, we’ll install it for you and use that version. You don’t have to wait for us, or manage binaries yourself. For example, if you want to use Hugo 0.20 right now, you can go to your site’s settings (Build and Deploy, Build Environment Variables section) and set HUGO_VERSION to 0.20 in your environment.

Basically, if you use netlify it will build with whatever version you tell it to. Otherwise you need to install specific versions locally. You can just drop the binary of the version you need in the root of that projects repository.

Which Version of Hugo Should I Run?

Scrolling back through the hugo releases when the HPSTR theme was in active development, I come across:

0.16.0 June 6th 2016

Hugo 0.16 is our best and biggest release ever. The Hugo community has outdone itself with continued performance improvements, beautiful themes for all types of sites from project sites to documentation to blogs to portfolios, and increased stability.

.deb packages for Debian, Ubuntu, etc.

Hugo has become part of the official Debian and Ubuntu repositories since January 2016!

That’s a nice note to find, since I’m an Ubuntu user.

v0.17 - October 2016

There are numerous options for different platforms, and this version of Hugo was released in October, one month after the final release of HPSTR.

Hugo is going global with our 0.17 release. […] Adding additional languages to your website is simple and straightforward.

Hugo continues its trend of each release being faster than the last.

Apparently it’s fast enough that people began using hugo’s webserver in production, around this time.

New in 0.17: Available as Snap package

Thanks to the contribution #2443 and guidance from @dholbach, Hugo is now available as a Snap package! (Snaps are a new kind of universal Linux packages.) Check it out at https://uappexplorer.com/app/hugo.hugo-authors

Now that I have Hugo working, lets see how far I can go in Hugo versions.

v0.30.1 - Oct 16, 2017

Hugo 0.30 is the Race Car Edition. Hugo is already very very fast, but much wants more. So we added Fast Render Mode. It is hard to explain, so start the Hugo development server with hugo server and start editing. Live reloads just got so much faster! The “how and what” is discussed at length in other places, but the short version is that we now re-render only the parts of the site that you are working on.

This is a big release, so I’m going to look ahead for the closest bug-fixes, but avoid any feature releases.

v0.31.1 - Nov 27, 2017

So.. there’s another big release after this, at the start of 2018, but I think I’ll stick with this one. It’s a year after HPSTR completed its development. I think we’ll be good here, and I want to get to know Hugo better before changing things up too much.

The next major change introduces page bundles. I wouldn’t be surprised if it has a problem with this theme.

Test Install

Once it’s installed, type I hugo version and read:

perfect!!

Clone the Hugo Theme.

With some pre-requisites out of the way, lets jump in at the first step in the theme setup:

$ mkdir newProject
$ cd newProject
$ mkdir themes
$ cd themes
$ git clone https://github.com/dldx/hpstr-hugo-theme.git hpstr

and you will see hpster located in: /web-work/themes/hpstr.

Example Site

Once you have the newProject/themes/hpstr you’ll find the folder exampleSite in the hpstr directory.

Just Copy the contents of exampleSite to the root of newProject, and test to see if it will run.

$ hugo

It should print something like this:

Started building sites ...
Built site for language en:
0 draft content
0 future content
0 expired content
10 pages created
0 non-page files copied
15 paginator pages created
9 tags created
0 categories created
total in 100 ms

Unless you get that print-out, don’t bother changing your whole sites configuration just yet.

Make sure you placed the contents of exampleSite into the root of your project directory, and that your directories are structured properly.

If you did everything right and it still won’t build, then I would shop around to different releases in that same time period to see if I could get one to work.

Just to be sure! I’ll test the server, also, and see that I get a website.

$ hugo server

Started building sites ...
Built site for language en:
0 draft content
0 future content
0 expired content
10 pages created
0 non-page files copied
15 paginator pages created
9 tags created
0 categories created
total in 61 ms
Watching for changes in /newProject/ {data,content,static,themes}

Yay!!!

Directory Structure HPSTR Jekyll vs Hugo

mmistakes.github.io/hpstr-jekyll-theme/theme-setup/

hpstr-hugo-theme/theme-setup

In with the new.

Now, it’s simply moving over the content, and swapping out some frontmatter, and configuration formating.

Content

Posts are stored in the content directory. By default, only content in the content/posts will show up in the All Posts section, however, you can link to other sections manually. For example, if you create a post at gallery/photo1.md, your post will appear both under the home page and under /gallery.

In Hugo-HPSTR-Theme, it’s all about your directory structure.

You’ll notice that your root directory mirrors the themes directory structure, because the theme always keeps a backup file of everything necessary to function.

archetypes/

I entered some tags and categories:

web-work/themes/hpstr/archetypes/default.md

+++
Description = ""
Tags = ["resources", "web-work"]
Categories = ["howto", "tools"]
menu = "main"
+++

data/

From YAML to TOML

In the theme directory is a navigation.yml file, and even awhole exampleSite that we can copy over to our root data directory and customize.

Be sure to change title: to title = and so forth.

[[links]]
title = "web-work.tools"
url = "/web-work.tools/"

[[links]]
title = "Services"
url = "/services/"

[[links]]
title = "Mostly Free SEO Tools"
url = "/seo-tools/"

[[links]]
title = "GitHub Pages Starter Pack"
url = "/github-pages-starter-pack/"

[[links]]
title = "Practical Public Key Crypto"
url = "/practical-public-key-crypto/"

[[links]]
title = "InfoMine"
url = "http://infominer.id/"

Frontmatter

The main differences:

  • using type rather than layout
  • using the frontmatter variable date to signify publication date, rather than hardcoding it into the title.

HPSTR-Jekyll

---
layout: post
title: "Sample Code Post"
description: "Examples and code for various HPSTR functions."
tags: [samples, code, snippets]
comments: true
link: http://mademistakes.com  
image:
  thumb: /images/pgp-og.png
  feature: pgp-banner.png
  background: triangular.png
modified: 2019-05-30T13:15:59-23:00
permalink: /sample-code/
---

HPSTR-Hugo

---
type: post
title: Sample Post
description: "Just about everything you'll need to style in the theme: headings, paragraphs, blockquotes, tables, code blocks, and more."
date: 2011-03-10

tags: [sample post]
image:
  feature: abstract-3.jpg
  credit: dargadgetz
  creditlink: http://www.dargadgetz.com/ios-7-abstract-wallpaper-pack-for-iphone-5-and-ipod-touch-retina/
---

config.toml

Now we are getting places! Next step is to copy the config.toml from the root of our example site into the root of our repository.

This is the final stretch, and we should be good to go


baseurl = "https://web-work.tools"
languageCode = "en-gb"
title = "web-work.tools: Independent Web-Work and Skyscraper Publishing."
theme = "hpstr"
pluralizelisttitles = false
PygmentsCodeFences = true
Paginate = 5
disqusShortname = "hpstrhugo"
publishdir = "docs"
enableEmoji = true
[params]
	subtitle = "Digital Tools for a Digital Transformation."
	[params.author]
		name = "⧉ Infominer"
		avatar = "/images/info-id.png"
		bio = "Full-Time Crypto-Curation and Histories ⧉ #Bitcoin #Blockchain #DecentralizedID ⧉ Research, Publishing, #WebWork #Indieweb ⧉"
		github = "webwork-tools/webwork-tools.github.io"
	[params.image]
	  feature = "/images/web-work-tools.png"



publishdir = "docs"

By setting this in our config.toml, we will be able to ask github pages to publish from the docs folder.

Redirects

One thing that’s important to note, if you are switching from Jekyll HPSTR to HPSTR Hugo with me. HPSTR Hugo makes all your posts live in the /posts/ directory.

If you used to let your blog live at the root of the site, like me, then add aliases, which are how Hugo does redirects.

aliases:
  - /title-goes-here/
  - /other-title-too/

One last thing!

It seems like you need to type hugo to publish to the docs directory, because hugo server doesn’t do both, it only does the webserving.. I don’t really understand, but that seems to be the case.

Use a linebreak before each list!

Twitter Cards

For twitter cards, I use the minimal-mistakes - _includes/seo.html as a reference for up to date syntax, and update the code that generates it wherever I need (or simply drop it in the includes of jekyll themes).

Test Branches

I’ve switched over to the Indieweb Hugo Theme, Indigo, a testament to how easy it is, here on Hugo.

Resources

Other Hugo Themes

Over the Rainbow