--- /dev/null
+---
+postid: 076
+title: May-June 2018 Shithub SSH key harvest, data and preliminary analysis
+date: July 7, 2018
+author: Lucian Mogoșanu
+tags: tech
+---
+
+As you may know if you'be been following the logs of The Most Serene
+Republic and the posts on Trilema[^1], RSA keys everywhere have been
+under the scrutiny of [The Supercollider][phuctor], and there's nothing
+anyone anywhere can do to stop that. And as it happens, I've been
+looking to add my humble incremental contribution on top of others', a
+contribution which consisted of two months of scraping SSH keys off the
+imperial hub of gits, also known as "GitHub".
+
+The **experimental methodology** consists of the following pipeline. On
+one side of the pipeline: a. start by querying the GitHub API
+[for users][github-all-users], i.e. getting
+`https://api.github.com/users?since=0` -- this will yield a list of
+users `U`; b. for each `u` in `U`, push the user's ID `u[i]` and
+name[^2] `u[n]` into a queue `Q`; c. take the highest user ID `maxi`
+found and head back to step (a), with `since=maxi` as an argument;
+repeat until there are no users left to process.
+
+On the other side of the pipeline: a. pop a `u` from `Q`; b. access
+`https://github.com/u[n].keys` to download a list of (newline-separated)
+keys `K`; c. for each `k` in `K`, process it and append it to a text
+file; repeat until the other side of the pipeline has nothing else to
+give us.
+
+Note that the side of the pipe described first, i.e. user enumeration,
+requires access to the GitHub API, which is rate-limited, which rate
+limit threshold can be increased by using an API key generated by an
+account. The other side of the pipeline can avoid calling the API
+altogether by getting `/username.keys`, which is pretty convenient[^3].
+
+So what does this **give us**? In theory, this gives us a list of SSH
+keys and some metadata, e.g. user names. In practice, moreover:
+
+[i] Some "users", e.g. `readme`, `session` and a few others, are in fact
+hard-coded to some page or the other. These are a minuscule part of the
+GitHub database.
+
+[ii] Sometimes, when `/username.keys` is under heavy usage, GitHub just
+throws a timeout message, probably due to a rate limit on database
+requests. This is easy to detect (the error page has a very specific
+layout) and retries can be automated or whatever.
+
+[iii] The last user ID registered on GitHub on cca. the 1st of July was
+`40,743,719`. That's how many users have been registered until then,
+some of them meanwhile deleted, and some of them... see below.
+
+[iv] For a large proportion of users, accessing `/username.keys` gives
+out a black-on-white page with the message "Not Found". What is this,
+then? Looking at the user names that yield this, one can notice that
+many of them look as if they were the product of a random string
+generator. So someone, probably more than one entity, is simply filling
+the GitHub database (and my poor text file) with spam accounts, which
+are then deleted using some spam filtering machine. It's obvious that
+the spam filtering mechanism is automated, because some of the users
+(e.g. user `40,743,719` above) were freshly registered and automatically
+banned. Nice, huh?
+
+[v] The number of "Not Found" spam-users amounted to a total of
+`7,747,033` at the time when I stopped, so about 20% of total
+registrations ever. Not bad for a bunch of spammers, I guess.
+
+[vi] The distribution of keys by type at the 1st of July is: 6426
+ecdsa-sha2-nistp256, 474 ecdsa-sha2-nistp384, 3729 ecdsa-sha2-nistp521,
+23408 ssh-dss, 33222 ssh-ed25519 and 4605938 ssh-rsa, or in a nice bar
+chart[^4]:
+
+<img align="middle"
+src="/uploads/2018/07/shithub-key-types.png">
+
+[vii] As per the figures above, there were only about 4.6 million RSA
+keys in existence on GitHub on the 1st of July 2018, as opposed to the
+approximately 6.9 million found by [JuroV][jurov] in his 2015 harvest. I
+can't really explain this. I'm pretty damn sure that I haven't missed
+anything, so I'll leave further discussion to the reader.
+
+This about sums it up. Now that the reader has gone through my
+not-so-short and possibly boring list, they can find the data
+[here][phathub-201806.csv.xz].
+
+~~~~
+$ sha512sum phathub-201806.csv.xz | gpg --clearsign -u lucian@mogosanu.ro
+-----BEGIN PGP SIGNED MESSAGE-----
+Hash: SHA512
+
+2a5aee998cfb33f1c2b1ebf2c0bdbe1f7a648c8d260e5a3a41f7cff0ecf97be0f33e5ca542e34aa7b8cfb41dc259c89809303c6a743cb2582ccec480fcf43813 phathub-201806.csv.xz
+-----BEGIN PGP SIGNATURE-----
+Version: GnuPG v1.4.10 (GNU/Linux)
+
+iQIcBAEBCgAGBQJbP1muAAoJEL2unQUaPTuVcXUQAKRnR/MUUaxi6uEcZVYVJA6s
+lGTSOM32gGXDXI3o+HidALDd2kaovIkOCxm2JAX08QtleQIbdrkquMlMHNNYBYh6
+bkqUSUo905LRwbuwkPP7pOKxR4zuwyj2XFFAH8hShe5ZKBgzbC+jBi21mwT8B7i9
+QsOtYGxwy3A3oLUAZDQaGZh59dGlJoRvQiJ3TX3m//rNczLqFd0DuIMfkAltjtj4
+Cyr0tgMY3wnQduXngRW+4TvIIq6lSV4b8n+vnG2ksKCdw/mfL/gCitLItA6F2XeL
+taSCZmLPn88so0XbLm55uEHiYsuaRCRKaqD+NcdZDZ5SJ6wiu01zfmgEHdmlNlGv
+ujB6NdDPCkFBOC/0l860/+f1G4z98DQMouM/A7AdeVJOEZCwPsnaZQS/FE3zbRWz
+v+PMj1nIsgG+kTMT01IoQn4c/vHgFxG+j4x7Y1vD0uF+iS3C0Dn7waFCVuD04dx9
+mhv0CPDhu7eWsCpM7+ilf5gpLAjg1oLjX7bGSJK/yCbBlN3gd6W/zt+fOMALrFb7
+WxN8JsAGGgP6VXI1nAj4xg75sV8gfdFHIz/kh/nmpaD1dmNkX7LTxc651ynQCMjd
+yzB2nmCQI3+CNVu6Gcd3JPthsLXDC2Qi6f1rkiYhWBgZbLZV8Tvhn/5ogshJJWEP
+uwFQM86fHrXTTBKr72Ma
+=ZYuq
+-----END PGP SIGNATURE-----
+~~~~
+
+**Update**, an archive with raw non-RSA keys:
+[github-non-rsa.tar.xz][github-non-rsa.tar.xz]. Each file in the archive
+is a set of three comma-separated values, where the first value is the
+user ID, the second the user name, and the third the SSH key.
+
+~~~~
+$ sha512sum github-non-rsa.tar.xz| gpg --clearsign -u lucian@mogosanu.ro
+-----BEGIN PGP SIGNED MESSAGE-----
+Hash: SHA512
+
+87a706b9649ebc1456c4331ac0225bd24b0b40fa72bc19af5164be090a15b09d10128e15e80daaa0ed55e6c2db6c474626e5629514d50798a73ee8b5dc107b9c github-non-rsa.tar.xz
+-----BEGIN PGP SIGNATURE-----
+Version: GnuPG v1.4.10 (GNU/Linux)
+
+iQIcBAEBCgAGBQJbQd7DAAoJEL2unQUaPTuVQHkQAKCgTVYP/2lSzgmLdQpXHETR
+/pSeBxAOp4FSjbqlipspnb+z8MLAUVs4l6kHyjdtjLK7B9KXi+lANLPXSOqAeMMQ
+gJm/T51Yy/JVJEsgWyoLwnw4XQcRX2kOxzpdpd+ERDOvB+d2Zzvw1amDUz2S6Z9C
+9QuIc005VJIZkMB+eC8uN7jb06leHcR6Q2bfB6WgbDAPvLBcrfRXGoZrmKAPgmrY
+Y22s1Kug1TnSaDEJs6m5QNzNLl9iDkU8o6aHgydYgtUDQSfN6jUZI0ZrdysEDxDf
+mu+jtHBgL/XoclaP08oy7LLhzWJ8qC0pA8Wixm26IRNQB2qWRXndWV5eCdtE2NNg
+4RtNfW9GF4pZ3hZSQ/ciDfWrRIK90sEzEHnd1Lw8vtIPVN3Naj3tDVNVYZBrIQDc
+7aK4QIIJTBuS5V2VvsiS1X/yKEZkJ9ZHMH4jBFIqQkwsbYYcTHT+UWY4iiqcqyHT
+DwpvksS1926xK4T8CF3IpwYknVhKa7l1MjC5/XVo6HdcSJ/DXJPUClqGoaBjoET3
+e+lP+eCXPiDzApb0vj+5SjgICgX4vRjupxc4lQR6o6C6qgW1OeZKUmBACgfIkymD
+P/c3iuxyeHY3hI5Q8kiT8qXt33vrwD2wLyHK1tnq0Pm+7gnOlQTCDMOYLA670mQ2
+F2IqZdc8aRMpU42RZWS8
+=iOmn
+-----END PGP SIGNATURE-----
+~~~~
+
+[^1]: And if you haven't, I wonder what you're doing here. No really,
+ I'm curious, you're invited [to explain][contact] if you
+ wish. Anyway, here's a bunch of references, for reference:
+ [1][phuctor], [2][logs-phuctor], and last but not least,
+ [3][trilema-cracked].
+
+[^2]: And whatever other metadata may seem useful. I for one haven't
+ found anything else of interest there.
+
+[^3]: For the harvester, definitely not for the GitHub git admins
+ maintaining their shitty infrastructure. What can I say,
+ [life's so unfair][trilema-licensing] when it's filled to the brim
+ with [hallucinated choice][trilema-hallucinated-freedom]. Or how did
+ that go?
+
+[^4]: Yes, the ordinate is a log scale, problem? It's not my fault that
+ RSA dominates the set by a few orders of magnitude.
+
+[contact]: http://webchat.freenode.net/?channels=#trilema&nick=from_thetarpit
+[phuctor]: http://phuctor.nosuchlabs.com/
+[logs-phuctor]: http://btcbase.org/log-search?q=phuctor
+[trilema-cracked]: http://trilema.com/2018/and-in-things-that-didnt-happen-today-heres-192-cracked-github-keys-some-hotties-in-tech-included-yes/
+[logs-spyked-shithub]: http://btcbase.org/log-search?q=from%3Aspyked+shithub
+[jurov]: http://explo.yt/post/2016/05/20/Parsing-OpenSSH-RSA-keys-in-Python
+[github-all-users]: https://developer.github.com/v3/users/#get-all-users
+[trilema-licensing]: http://trilema.com/2015/a-new-software-licensing-paradigm/#selection-136.0-136.1
+[trilema-hallucinated-freedom]: http://trilema.com/2017/the-practical-costs-of-hallucinated-freedom/
+[phathub-201806.csv.xz]: http://lucian.mogosanu.ro/randomio/phathub-201806.csv.xz
+[github-non-rsa.tar.xz]: http://lucian.mogosanu.ro/randomio/github-non-rsa.tar.xz
--- /dev/null
+---
+postid: 077
+title: A photographic tour of Bucharest; in today's issue: Lacul Morii
+date: July 8, 2018
+author: Lucian Mogoșanu
+tags: in-the-flesh
+---
+
+So this Wednesday I grabbed my fancy-schmancy DSLR and went to another
+not-quite-aimless stroll through town. Seeing that the weather finally
+took a break from the daily thunderstorms[^1], I headed off to the
+Western part of the town, in the Crângași neighbourhood, where a human
+construction called Lacul Morii is situated. The result was quite
+surprisingly ninety photos, of which I processed twenty, of which
+fifteen or so are posted below.
+
+<a href="/uploads/2018/07/lacul-morii-0005.png"> <img align="middle"
+class="thumb" src="/uploads/2018/07/lacul-morii-0005-thumb.png"> </a>
+
+Speaking of below, above: the campus of University Politehnica of
+Bucharest, one of Romania's best schools, from which emerged well-known
+names such as Ion Iliescu, Ecaterina Andronescu, Petre Roman and Liviu
+Dragnea. Oh, and they also teach engineering there, or so they
+say. Anyway, just in case you're wondering, the UFO-shaped oddity in the
+background is the last level of the Rectorate building, and we[^2] are
+all waiting for the day when it takes off. When -- not if! occult Dacian
+technology and all that.
+
+A bit to the north, the Dâmbovița river[^3] separates the academic
+campus, i.e. the place where students supposedly go to study, from the
+Regie neighbourhood and student campus[^4], where students go to get
+some sleep now and then, but primarily for eating, partying, fucking,
+camwhoring -- in other words, living. Too bad the place is also filled
+with massive amounts of huge cockroaches. Anyway, closing this
+particular parenthesis:
+
+<a href="/uploads/2018/07/lacul-morii-0013.png"> <img align="middle"
+class="thumb" src="/uploads/2018/07/lacul-morii-0013-thumb.png"> </a>
+
+Below, a wide view of Lacul Morii. And this is where, predictably, I stop
+again to tell a story.
+
+<a href="/uploads/2018/07/lacul-morii-0019.png"> <img align="middle"
+class="thumb" src="/uploads/2018/07/lacul-morii-0019-thumb.png"> </a>
+
+Back in the nineteenth century, when Bucharest was way more of a
+shithole than it is today, "nobody could have predicted" that the
+Dâmbovița river would frequently flood the town, not to mention the
+smelly smells and the general swamp atmosphere -- where somewhat
+unrelatedly, somewhere on the river one could find the Ciurel
+Watermill. Anyway, in the late twentieth century, an illiterate shoeman
+-- meanwhile [shot][july-theses] by a bunch of other illiterate "people
+themselves" -- in his destructive march to redesign the city, rebuilt
+the riverbed on two levels, one for the river and one for water
+recirculation through the town's sewers, and built the foundation for
+the so-called "Mill Lake", connected to the river via a dam to control
+the flow[^5].
+
+That aside, the lake is surrounded by a pier, where in the evenings the
+local fauna goes for a walk, running, fishing, the occasional wank in
+the bushes, and... well, what are these guys up to anyway:
+
+<a href="/uploads/2018/07/lacul-morii-0021.png"> <img align="middle"
+class="thumb" src="/uploads/2018/07/lacul-morii-0021-thumb.png"> </a>
+
+In the northern part of the lake, an island:
+
+<a href="/uploads/2018/07/lacul-morii-0030.png"> <img align="middle"
+class="thumb" src="/uploads/2018/07/lacul-morii-0030-thumb.png"> </a>
+
+<a href="/uploads/2018/07/lacul-morii-0034.png"> <img align="middle"
+class="thumb" src="/uploads/2018/07/lacul-morii-0034-thumb.png"> </a>
+
+Above, the Ducks come from the Trucks -- I heard that on TV, so it must
+be true! Below, a shot idly taken from under a tree, where the
+afternoon's blazing sun couldn't reach me.
+
+<a href="/uploads/2018/07/lacul-morii-0036.png"> <img align="middle"
+class="thumb" src="/uploads/2018/07/lacul-morii-0036-thumb.png"> </a>
+
+At twenty meters' distance on the shore there was a guy trying to read
+a book, while being pestered by an old bum who yammered about some TV
+starlet or another, and complained about how young girls nowadays keep
+those fat cunts of theirs in their blue jeans all day long on this hot
+weather, letting them gather melted cheese, and why don't they let the
+pussy get some fresh air anyway? Poor idiot had plenty of decades to
+figure out why the social state sucks: it gives him all that money for
+nothing, but no chicks for free. And yet he didn't, and why would that
+be? Would you happen to know?
+
+<a href="/uploads/2018/07/lacul-morii-0042.png"> <img align="middle"
+class="thumb" src="/uploads/2018/07/lacul-morii-0042-thumb.png"> </a>
+
+Above, a bunch of plants from (I can only guess) the Carduus genus,
+known around the place as "ciulini", widespread in this geographical
+region, ready to further spread their seeds by way of winds. Below, a
+willow tree shielding the lens from the sunlight; also, a pair of
+leftover plastic bottles on the ground, because what, Romanians around
+these places are orcs.
+
+<a href="/uploads/2018/07/lacul-morii-0048.png"> <img align="middle"
+class="thumb" src="/uploads/2018/07/lacul-morii-0048-thumb.png"> </a>
+
+<a href="/uploads/2018/07/lacul-morii-0058.png"> <img align="middle"
+class="thumb" src="/uploads/2018/07/lacul-morii-0058-thumb.png"> </a>
+
+Above: look ma, I r arteest! Below: moar arteesteec geometry, followed
+by a coupla more lake shots.
+
+<a href="/uploads/2018/07/lacul-morii-0061.png"> <img align="middle"
+class="thumb" src="/uploads/2018/07/lacul-morii-0061-thumb.png"> </a>
+
+<a href="/uploads/2018/07/lacul-morii-0069.png"> <img align="middle"
+class="thumb" src="/uploads/2018/07/lacul-morii-0069-thumb.png"> </a>
+
+<a href="/uploads/2018/07/lacul-morii-0071.png"> <img align="middle"
+class="thumb" src="/uploads/2018/07/lacul-morii-0071-thumb.png"> </a>
+
+Finally, that damn dam. The writing on that wall pre-dates 1989, which
+is easy to discern by the average Romanian for orthographic
+reasons. "River" is cognate with "rivus", which led to "râu", right? So
+then, why the fuck "â" and not "î"? Because a bunch of wretches known as
+"the Romanian Academy" imagined themselves the navel of Romanian
+speaking and writing, is why.
+
+<a href="/uploads/2018/07/lacul-morii-0076.png"> <img align="middle"
+class="thumb" src="/uploads/2018/07/lacul-morii-0076-thumb.png"> </a>
+
+Post-finally, and further downstream: a night pic of a river; and a
+night pic of a road that leads to a river, crossed by a bridge that
+leads nowhere in particular. Bonne nuit, Paris d'Orclandie!
+
+<a href="/uploads/2018/07/lacul-morii-0080.png"> <img align="middle"
+class="thumb" src="/uploads/2018/07/lacul-morii-0080-thumb.png"> </a>
+
+<a href="/uploads/2018/07/lacul-morii-0091.png"> <img align="middle"
+class="thumb" src="/uploads/2018/07/lacul-morii-0091-thumb.png"> </a>
+
+[^1]: Oh, by the way. This week there was a jazz festival organized in
+ ye olde Bucharestian shithole, and on the taxpayer's money, no
+ less. All this after on last Sunday, while taking a similar stroll
+ through the Cișmigiu gardens, I stumbled upon an "international
+ folklore festival", also organized by the local authorities, where
+ one of the headliners were none other than the famous Romanian
+ ethno-blues group Nightlosers. Who -- by the way, if you're still
+ living under the delusion that "[nations][desteapta-te-romane]" are
+ still a thing -- are formed of a German vocalist/guitarist, two
+ Magyars (the violinist and the keyboardist) and two "Romanians": an
+ Ardelean drummer from Cluj and... well, I don't know anything about
+ the bassist to be honest.
+
+ Anyway, this Saturday was the second-to-last day of a jazz festival
+ where this pretty great Norwegian guy Jan Garbarek played the
+ saxophone, and it rained like hell all the afternoon, because why
+ the fuck not. And sure, I went to the concert anyway -- but if I
+ hear anyone blaming this on "global warming" and whatnot, I promise
+ to warm you up with this very nice baseball bat that I have.
+
+[^2]: UPB graduates, that is.
+
+[^3]: The [very same][botanical-garden], yes.
+
+[^4]: Its name coming from "Regia Monopolurilor Statului",
+ i.e. something along the lines of "The Direction of State
+ Monopolies", which, I'll venture a guess, was built there because
+ the area was one of the
+ [more industrialised ones][botanical-garden-ii] in town at the time.
+
+[^5]: So if you happen to live in Bucharest, you now know whom you have
+ to thank for the fact that the town sucks a bit less -- lu'
+ împușcatu', bă nene, bă. And yes, I'm pretty sure that much of the
+ work involved in making this happen is now a lost art, given that
+ "[they don't use them for nuthin' anyway][education]". So at some
+ point your children will be forced to reap the fruits of your
+ laziness, and they'll either be smart enough to reinvent the wheel,
+ or they'll die of hunger, poverty, hallucinated options and all
+ that. Yes-yes, all that because you're "tânăr, frumos și liber",
+ because "you just wanted to" give your children a "nice place" to
+ live, not stopping to think for even a moment that the only way to
+ come to have a nice place to live is through sweat, blood and
+ tears. Da, coaie, this is all on you.
+
+[desteapta-te-romane]: /posts/y03/054-desteapta-te-romane.html
+[botanical-garden]: /posts/y04/070-bucharest-botanical-garden.html#selection-312.0-312.1
+[botanical-garden-ii]: /posts/y04/070-bucharest-botanical-garden.html#selection-44.0-44.1
+[july-theses]: /posts/y03/05a-july-theses.html#selection-901.0-901.109
+[education]: /posts/y03/055-on-education.html#selection-81.252-85.0
projects / thetarpit.git / commitdiff
