Submit a new text post

DarkNetMarkets

created by [deleted]

0
270

Dark Net Market archives, 2011-2015 (self.darknetmarkets)

submitted by gwern

I am releasing all my DNM scrapes: a 50GB (~1.6TB) collection covering 89 DNMs & 37+ related forums, representing <4,438 mirrors.

It can be downloaded [as a torrent](https://archive.org/download/dnmarchives/dnmarchives_archive.torrent); for some torrent clients, the torrent may not work for various reasons, in which case you can download everything [directly from the IA](https://archive.org/download/dnmarchives). But for all the other details, see [my writeup on the contents](http://www.gwern.net/Black-market%20archives).

A quick overview of what is covered:

- Markets:

- 1776, Abraxas, Agape, Agora, Alpaca, AlphaBay, Amazon Dark, Anarchia, Andromeda, Area51, Armory, Atlantis
- BlackBank Market, Black Goblin, BlackMarket Reloaded, Black Services Market, Bloomsfield, Blue Sky Market, Breaking Bad, bungee54, BuyItNow
- Cannabis Road 1, Cannabis Road 2, Cannabis Road 3, Cantina, Cloud9, Crypto Market / Diabolus,
- DarkBay, Darklist, Darknet Heroes, DBay, Deepzon, Doge Road, Dream Market, Drugslist
- East India Company, Evolution, FreeBay, Freedom Marketplace, Free Market, GreyRoad, Havana/Absolem, Haven, Horizon, Hydra
- Ironclad, Kiss, Middle Earth, Mr Nice guy 2, Nucleus, Onionshop, Outlaw Market, Oxygen
- Panacea, Pandora, Pigeon, Pirate Market, Poseidon, Project Black Flag
- Sheep, Silk Road 1, Silk Road 2, Silk Road Reloaded (I2P), Silkstreet, Simply Bear
- The BlackBox Market, The Majestic Garden, The Marketplace, The RealDeal, Tochka, TOM, Topix 2, TorBay, TorBazaar, TorEscrow, TorMarket, Tortuga 2
- Underground Market, Utopia, Vault43, White Rabbit, Zanzibar Spice
- Forums:

- Abraxas, Agora, Andromeda, Black Market Reloaded, BlackBank Market, bungee54, Cannabis Road 2, Cannabis Road 3
- DarkBay, Darknet heroes, Diabolus, Doge Road, Evolution, Gobotal, GreyRoad, Havana/Absolem, Hydra, Kingdom, Kiss
- Mr Nice Guy 1, Nucleus, Outlaw Market, Panacea, Pandora, Pigeon, Project Black Flag, Revolver
- Silk Road 1, Silk Road 2
- TOM, The Cave, The Hub, The Majestic Garden, The RealDeal, TorEscrow, TorBazaar, Tortuga 1
- Underground Market, Unitech, Utopia
Miscellaneous:

- Assassination Market
- Cryuserv
- DNM-related documents (primarily the Ross Ulbricht trial evidence exhibits)
- DNStats
- Grams
- Pedofunding
- SR2doug's leaks

I hope it will be useful.

---

In other news, I am stepping down on /r/DarkNetMarkets mod. I have been planning to do so since the beginning of the year, but I wanted to finish up some big DNM projects before I cut more ties: the first was [my arrests/bust compilation](http://www.gwern.net/Black-market%20arrests) and the second is this, releasing the full scrape archive which I've been working on for years.

I'm no longer as interested in the DNMs as I used to be. I was originally lured in by the fascination of watching a small cryptopunk revolution, and I was hopeful that it would go beyond the SR1 model into multisig and beyond. But we have seen little multisig usage, and that augurs ill for the distributed marketplaces. (What if someone built a trustless distributed blackmarket and no one wanted to use it?) Instead, the SR1 business model has proven remarkably durable despite the constant wearying turmoil of exit-scams and hacks and as I write this in mid-July 2015, there seems to be little chance that things will change substantially soon. (If Tor hidden services are cracked, the DNMs will simply move to I2P or Freenet; if Bitcoin starts getting too easily tracked, transactions will be settled in Monero or another cryptocoin with better anonymity properties, etc.) So I'd like to stop investing so much energy into monitoring DNMs, which includes stopping my scrapes and stepping down as moderator.

The exact timing owes something to the LE attention on me; as unpleasant as the ICE subpoena was, I was almost as disturbed by the subsequent Coinbase persecution of people who had done little but donate some Bitcoin to me. (And more recently, an order of some electronics from China arrived with the box open and the packaging so loosely taped one could get one's finger inside it; this *could* have simply been lazy packing or caused by squishing en route, but on the other hand, international packages at the border have few or no protections so it would be easy to flag a particular name for closer inspection...) I would have liked to finish up earlier, actually, but it's taken a lot of effort to get the archive into a releasable state (just uploading the 52GB to the IA took ~3 days on my connection).

I did not do much in the way of day to day moderation on /r/DNM, so there will be little observable change; the current mod team minus me will be pretty much the same as it was before. I also will probably continue to comment, offer advice, or post from time to time when something interesting comes up; at some point, I would like to finish my overview of proposed distributed market systems from a DNM perspective, and try correlating DNM downtime against the Bitcoin exchange rate.

But otherwise, I am mostly done.

Posting this archive will be my last action as DNM moderator.

It's a good archive, I think.
Sorting through it, I refound a lot of things I'd forgotten: the BMR mods chatting on Pastebin about one of them being arrested, unaware that Dutch police were watching them and tapping all their phones and they would be going to jail for years in weeks/months, taking down Utopia with them; Backopy bristling at my criticisms of BMR security while preparing to take down BMR for being insecure; Ross and Rene at StoryTime, totally high, talking about Julia (possibly neither of them will ever see Ross again as he's sent to prison in Virginia for life) and Ross's twitter making an attempt at technical analysis of Bitcoin exchange rates (he sounds competent, but it's rather undermined by how the Mtgox leaks showed he sold off a huge amount of bitcoins not long before Bitcoin made some of its biggest gains ever) and his drawings (totally psychedelic); my crawls of The Marketplace, which never got the credit it deserved for pioneering multisig and tends to be left out of the post-SR1 story; the evidence that Sheep was run by Tomas Jirikovský, which I got so much shit for releasing; all the markets with funny themes like Area51 and Absolem and - of course! - Breaking Bad; even the worst markets like Black Goblin are still amusing to think back on; the Ulbricht trial exhibits where they forgot to redact *both* copies of information and so wound up leaking information about Force/Bridges and enabling some journalists to ransack DPR's email accounts (they never did release all of what they found, the greedy bastards); and photos of drugs, of every sort, from every market and vendor, drugs you didn't know existed or were still made, etc etc...

All these and more I give you.

I give you back your memories.

all 110 comments
sorted by:
best

The_Grid_Is_Up56 points  

Farewell Gwern, you shall be missed.

IGetDankShit37 points  

:'(

It's been an honor having you as a moderator and we'll certainly miss your contributions to this community. Keep us updated on future projects. I'd still like to help fund a book on the DNMs written by you if that's something you ever become interested in. Much love and respect to you gwern, you're one of a kind.

pxck25 points  

Thank you based gwern

Jay-__22 points  

Like I already said; I hope you'll find something worthwhile to do soon. You'll surely be missed around here. :/

Rectungle22 points  

A sad day for the DNM community indeed, you will be dearly missed

BoxAddict14 points  

I feel like this post deserves a montage or a slideshow with music. So much reminiscing and joyful sadness.

SecondWarning12 points  

Take care, Gwern. Thanks for your help moderating this sub.

boxofpopcorn11 points  

Thank you Gwern, for all of the popcorn.

ShulginsCat10 points  

> my overview of proposed distributed market systems from a DNM perspective

What's this about? Can you give some more details what you are thinking here?

gwern12 points  

It's straightforward. I've drawn up a list of all the proposals I could find, and thought hard about what pieces a distributed DNM needs. What I need to do next is read through all the design docs for the various proposals and see how well I think they implement the necessary pieces.

ShulginsCat6 points  

Could I trouble you to share the list of proposals you found? (with or w/o your commentary is fine)

gwern12 points  

My current list of possibilities goes:

- Bitwasp
- Bitbay
- MaidSAFE
- NXT
- DarkMarket/OpenBazaar (fork: https://github.com/openmarket/openmarket ?)
- P2POX http://bxroberts.org/p2pox/DESIGN.html and http://bxroberts.org/p2pox/USE_CASE.html
- Shadow Market http://shadowcash.info/display/shadow/Welcome+to+Shadow http://motherboard.vice.com/read/the-race-for-the-first-decentralised-silk-road-is-on
- Syscoin
- Bitmarkets http://voluntary.net/bitmarkets/whitepaper/
- Slur http://slur.io/ , Darkleak
- MayDay http://thehub7gqe43miyc.onion/index.php?topic=7950.0
- Nashx http://nashx.com/HowItWorks
- SafeMarket http://safemarket.github.io/ https://www.reddit.com/r/Bitcoin/comments/3b7ijs/safemarket_a_trustless_decentralized_pseudonymous/

(See also https://en.bitcoin.it/wiki/Distributed_markets )

ShulginsCat7 points  

After a quick run-through I think Axis-Mundi (MayDay) has the best chance of being a winner. Although not a true decentralized market due to the requirement that some people run Broker nodes, it is the only one that runs over Tor and was built from the ground up to meet the requirements of buying and selling drugs anonymously.

Here's a clearnet link for the curious:

https://six-pack.github.io/Axis-Mundi/

Fatvod1 point  

I disagree. I just see some slapped together protocols. I really like the way openbazaar talks about the theory behind what they are doing, they seem to know more about algorithm and transaction handling than most.
https://gist.github.com/drwasho/405d51bd1b1a32e38145

ShulginsCat1 point  

Sure, get OB to work over Tor and we'll have something to talk about.

Fatvod1 point  

https://www.reddit.com/r/OpenBazaar/comments/2iusp4/openbazaar_tor_demo/

that was 9 months ago

ShulginsCat2 points  

*eating my hat*

ShulginsCat1 point  

I looked into this a bit. Two things:

1. One of the developers for the Tor port (El Presidente) is now a developer for Axis Mundi.

2. The [pull request for Tor support](https://github.com/OpenBazaar/OpenBazaar/pull/860) was never merged and the last comment is: "Closing this as it will most likely no longer work with rUDP. For now."

Fatvod1 point  

Hmm. Interesting. The developers have been a bit quiet recently. I wonder what their plans are.

-El_Presidente-2 points  

You are being a little unfair to Axis Mundi, it is quite simplistic in it's design using only a single protocol, MQTT - designed for the Internet of Things no less. It uses the same Ricardian contract concepts as OB (similar at least) but is focused on maintaining privacy and anonymity at all times. We have high hopes for that project.

As for the demo you mention, that was developed, built an run by us - as was this Pull request implementing full native Tor support into OB last year https://github.com/OpenBazaar/OpenBazaar/pull/860

It worked very well in fact.

However OB has moved on a lot architecture and network wise and will take some additional work now to get it Tor friendly again. It's certainly possible but problems like having to be online all the time and having to run a hidden service need to be addressed - it's not that they can't be, they just need to be.

OB is a great piece of software and it has a strong following. Perhaps with funding now in place it may bemore troublesome for them to allow the Platform to be used in a DNM context.

Jay-__1 point  

OT, but are you the guy that runs the 'All Markets Vendor Directory'?

I'm almost sure I already asked this once..

MrGangGreen1 point  

What's happened with the syscoin market? That shit looked promising.

The__Hans1 point  

The most projects are very good at PR but very slow at developing. I consider Syscoin one of them. Instead of publishing what they announced they opened a merchandise store and found a millionairs club.

The__Hans1 point  

I compile a list too. [At the psychonautwiki.](https://psychonautwiki.org/wiki/Comparison_of_darknet_markets#Decentralized_Markets ) It certainly needs more references. I think Ill need to add some projects now that I see you list. Feel free to contribute. If you can not edit the page pm [this guy](https://psychonautwiki.org/wiki/User:PJosepherum).

BinaryResult2 points  

Have you seen [Drop Zone](https://www.reddit.com/r/Bitcoin/comments/30q9ou/a_proposal_for_a_100_decentralized_darknet_market/)?

gwern2 points  

I think I did, but it struck me as completely hopeless. Dead drops simply do not work as a delivery mechanism.

ShulginsCat2 points  

... except in Russia, for some reason

gwern5 points  

RAMP only did that in Moscow, I heard, and only occasionally then. Which makes sense. You may be able to get a functioning market with extreme density and lots of drug users. You might be able to get it up and running in Manhattan, San Francisco, and a handful of other high-drug-use high-density cities, but everywhere else? Globally? No frigging way.

BinaryResult2 points  

We need a decentralized drone network :)

select1on8 points  

ohhh wow this is cool, I want to see what's in there for Dnstats. I just bought DarknetMarket.com so I might make it public on there as some kind of archive.

We will miss a real asset here. Hope you pop in now and then. Take care.

## I don't have much but I donated a bit over 1btc for your efforts. I'd give more if I can.
Just read your write up, Appreciate the comments. Cheers mate.

gwern6 points  

I'm afraid it's just the export you provided me, nothing beyond that. I could throw up some of the R code I was using to take a peek at the effect of Operation Onymous on downtime but otherwise, you have all the data I do.

select1on5 points  

Cool - thanks for everything.

select1on3 points  

Is it possible to get a copy of your scraping scripts? I might continue if it's just a matter of space and bandwidth.. Just a thought.

boxofpopcorn3 points  

I have longed for the source code to Gwern's crawling scripts. Please become a reality.

gwern7 points  

The Privoxy blacklist is in the `documents.tar.xz` archive. The actual scripts are just really long shell commands, and I gave one in my writeup for Crypto Market in the http://www.gwern.net/Black-market%20archives#how-to-crawl-markets section, explaining what each part does. All the others are just slight variants.

Agora forums looks like this:

cd /home/gwern/blackmarket-mirrors/agora-forums/ && rm ./cookies.txt; fgrep --no-filename '.onion' ~/cookies.txt ~/`today`/cookies.txt > ./cookies.txt; http_proxy="localhost:8118" wget --mirror --page-requisites --load-cookies=cookies.txt --keep-session-cookies --user-agent="$(cat ~/blackmarket-mirrors/user-agent.txt)" --retry-connrefused --waitretry=10 --read-timeout=200 --timeout=150 --append-output=log.txt --server-response 'http://lacbzxobeprssrfx.onion/index.php/topic,62888.0.html' 'http://lacbzxobeprssrfx.onion/index.php?action=profile;u=55095' 'http://lacbzxobeprssrfx.onion/index.php?topic=64373.0' 'http://lacbzxobeprssrfx.onion/index.php?action=recent' ; mv lacbzxobeprssrfx.onion/ `today` ; mv log.txt `today`; rm cookies.txt # Agora forums

Agora market looks like this:

cd ~/blackmarket-mirrors/agora/ && rm ./cookies.txt; fgrep --no-filename '.onion' ~/cookies.txt ~/`today`/cookies.txt > ./cookies.txt; http_proxy="localhost:8118" wget --mirror --page-requisites --load-cookies=cookies.txt --keep-session-cookies --user-agent="$(cat ~/blackmarket-mirrors/user-agent.txt)" --retry-connrefused --waitretry=1 --read-timeout=20 --timeout=15 --max-redirect=0 --wait=2 --append-output=log.txt --server-response 'http://agorahooawayyfoe.onion/cat/vheN9d2mMr' ; mv ./agorahooawayyfoe.onion/ `today` ; mv log.txt `today`; rm cookies.txt; alert # Agora / agora

A few have tweaks, like a loop I wrote for Outlaw back when it was hard to mirror for some reason so I simply downloaded every possible item ID:

cd ~/blackmarket-mirrors/outlawmarket/ && rm ./cookies.txt; fgrep --no-filename '.onion' ~/cookies.txt ~/`today`/cookies.txt > ./cookies.txt; http_proxy="localhost:8118" wget --mirror --retry-connrefused --waitretry=1 --read-timeout=20 --timeout=15 --tries=10 --load-cookies=cookies.txt --user-agent="$(cat ~/blackmarket-mirrors/user-agent.txt)" --append-output=log.txt --server-response 'http://outfor6jwcztwbpd.onion/index.php?list' & for i in {1..5000}; do (http_proxy="localhost:8118" wget --page-requisites --retry-connrefused --waitretry=1 --read-timeout=20 --timeout=15 --tries=10 --load-cookies=cookies.txt --user-agent="$(cat ~/blackmarket-mirrors/user-agent.txt)" --append-output=log.txt --server-response "http://outfor6jwcztwbpd.onion/index.php?item=$i"); done; mv outfor6jwcztwbpd.onion/ `today`; mv log.txt ./`today`/; rm cookies.txt # outlaw market / Outlaw Market

Similarly, to iterate over the Alphabay users:

cd ~/blackmarket-mirrors/alphabay/ && rm ./cookies.txt; fgrep --no-filename '.onion' ~/cookies.txt ~/`today`/cookies.txt > ./cookies.txt; for i in {1..70000}; do (http_proxy="localhost:8118" wget --retry-connrefused --waitretry=1 --read-timeout=20 --timeout=15 --tries=10 --load-cookies=cookies.txt --user-agent="$(cat ~/blackmarket-mirrors/user-agent.txt)" --append-output=log.txt --server-response --max-redirect=0 "http://pwoah7foa6au2pul.onion/user.php?id=$i"); done; mkdir `today`; mv ./user.php* log.txt ./`today`/; rm cookies.txt # Alphabay user crawl

Ugly, I know. What can I say? They slowly evolved, option by option, copy-paste by copy-paste. It was never the right time to write a custom scraper which could do things better.

tgif32 points  

> much but I don

I'd love the R code as you know from previous pm's haha R frustrates me

gwern4 points  

I think this should work but it's been a few weeks since I last touched it:

dnmUptime <- read.delim("dnstats-20150712.sql", na.strings="NULL",
nrows=6000000, colClasses=c("factor", "factor", "factor", "integer",
"factor", "numeric", "numeric", "POSIXct"))
markets <- dnmUptime[dnmUptime$type==1,]
dnmUptime <- NULL # save RAM
markets$Date <- as.Date(markets$timestamp)
markets$Up <- markets$httpcode == 200
daily <- aggregate(Up ~ Date + sitename, markets, mean)
library(ggplot2)
qplot(Date, sitename, color=Up, data=daily) + geom_vline(xintercept=as.Date("2014-11-05"), color="red")

tgif31 point  

Thanks bud. More for me to learn!

natenorman8 points  

So much has happened this year in this community it amazes me. I've seen so much change in a short time, its almost as if I'm coming towards the end of the party. I can't really say where it will go from this point as usual you made some very fine points about stability and change in the community which probably will end up being true.

As soon as I saw this post I knew it was a retirement post, so many mods have left recently I knew you would end up leaving too. I saw your posting get lighter and lighter, with slight hints of frustration in them. Honestly even though you're leaving this whole board owes you.

I'm going to keep it short, thanks Gwern; you made the DNM's safer for everyone.

drimilr7 points  

wow thanks for this! and everything else!! your market analysis is why i became curious, and eventually, albeit casually, stepped into the markets.

good luck! and hope to read more of your analysis down the line; darknet, or otherwise (i might have tried n-back because of your site too. and no, i dont think i got any better at it)

ciphersexual7 points  

THANKS. Will you continue to moderate what remains of /r/silkroad?

gwern5 points  

Yeah, probably. It's easy and doesn't require me to be constantly up to date.

IGetDankShit6 points  

Honestly that sub could really use some more active moderation. Take the [top post](https://www.reddit.com/r/SilkRoad/comments/3dg79q/xanax_man/) there right now for example, the OP of that post was banned from this subreddit almost 3 months ago, and almost every single reply in that post is one of his shill accounts trying to trick people into thinking it's legit. A few minutes ago it looks like someone almost fell for the scam, hopefully he'll see my message and do a little more research before losing his money. So yeah, the sub needs more active moderation. I think you should stop allowing new posts to be submitted or manually approve all posts yourself, or find someone willing to do that dirty work. Right now that subreddit is a scammer's paradise and nobody is doing anything about it.

gwern3 points  

You're right, but I'm hesitant to shut down submissions entirely. That's a lot more heavy-handed than I like and I don't check the submission queue every day. (Or every week.) If you want to commit to checking it regularly, I could make you a mod and then set submissions to moderation.

IGetDankShit2 points  

That's not a responsibility I want to take on right now, but I do feel it is important that you do *something* about it, which is why I wanted to bring it to your attention. I know disabling submissions is not ideal, but it's a much better alternative than the status quo. Hopefully you can find someone who has the time to keep that subreddit safe for the subscribers.

KimJongUntouchable1 point  

Change the CSS to point them toward this sub.

ShulginsCat2 points  

Why does that sub still exist?

gwern3 points  

It exists because some people are interested in SR1/2 topics specifically, and that news tends to get buried in the firehose of /r/DNM. So if you subscribe to /r/silkroad, you can hopefully learn about the latest news in the Ulbricht trial or what will happen to Libertas or old sellers finally being busted or new analyses of SR1/2 data, etc, without thousands of posts about the latest DNM drama or sales or new vendors or whatever.

ShulginsCat1 point  

I see the value, but the name has too much scope. Maybe /r/DarknetArrests ?

IGetDankShit2 points  

Perhaps you could sticky a post to the top of the subreddit which tells people that the subreddit is not being actively moderated and to be careful and vigilant of scams, and to always check /r/DarkNetMarkets for more information. Something to give unsuspecting people a little bit of a heads up.

tgif36 points  

As a service to the community. This will be seeded on a 1gb line seedbox for a week and if it is still active will remain until leechers lower.

Ps. Torrent is an ok idea just I can see the ip of everyone leeching it from me. Then again this is just public info for research.

YounginLam5 points  

peace out gwern. never forgotten

FreeTradeCocaine5 points  

Goodbye Gwern, thank you for all the interesting posts and your contributions to this sub!

dnm2015 points  

Thank you for all you have done! You will be missed dearly. Much love and respect!

culdesacked5 points  

Your write up on your site is impressive, well done. Thanks for all your contributions on DNM.

Munchie_King5 points  

Very rarely do I save submissions on reddit. This is pure gold. Thank you for everything you've done and taught - I've actually learned a lot from you. With that being said, I hope you have success with whatever future endeavors you may have.

dsfadsfavxzcv5 points  

Gwern, you are an amazing man in many ways.

I feel shitty asking you for one last thing, but could you explain how one might archive /r/darknetmarkets and /r/agmarketplace and how to set up those again if they are ever deleted/banned? Maybe on voat or somewhere else, whatever you think is feasible.

gwern3 points  

I believe Archive Team has a Reddit-archiving bot. You could try asking at https://www.reddit.com/r/Archiveteam/ or http://archiveteam.org/index.php?title=Reddit

coffeencreme5 points  

Gwern I've been here in different incarnations, lurked and posted, and your work has always been amazing.

Good luck in whatever you do next.

2ndaxct4 points  

I have been lurking for over 5 months now. You have always offered valuable insight about the DNM's to me, as someone who had no idea about anything.

This is the first time i reply to a post by you. It will be sad seeing you leave, but great that you realized you have to move on.

Good luck with your next venture, it will be as great as your contributions if not more.

TotesMessenger4 points  

I'm a bot, *bleep*, *bloop*. Someone has linked to this thread from another place on reddit:

- [/r/deepdotweb] [Dark Net Market archives, 2011-2015 : DarkNetMarkets](https://np.reddit.com/r/DeepDotWeb/comments/3dgyo1/dark_net_market_archives_20112015_darknetmarkets/)

- [/r/silkroad] [SR1 &amp; SR2 scrapes now available](https://np.reddit.com/r/SilkRoad/comments/3dg0oy/sr1_sr2_scrapes_now_available/)

[](#footer)*^(If you follow any of the above links, please respect the rules of reddit and don't vote in the other threads.) ^\\([Info](/r/TotesMessenger/wiki/) ^/ ^[Contact](/message/compose/?to=\\/r\\/TotesMessenger))*

[](#bot)

simplemindedslut4 points  

Thank you for all your efforts. It's rare to have a voice of reason in these murky waters. This is a sad day for our community.

passengerB4 points  

The darknet markets would not be where they are today without you, gwern. You are the master of a very complex subject and I'm thankful of all your time, effort and contribution to the community.

Good luck, and here's to a prosperous future.

Goldmedal_4 points  

5-10 years from now, I hope that things are interesting enough to entice you to come back again. I too am getting bored with DNM's development, but have high hopes for the future.

q2x4 points  

Gwern is a light in the darkness.

torr0t4 points  

Thanks for everything Gwern. Downloading the archive now, I can't wait to look through it. I will always keep it as a backup anyways. All the best.

Litecoin_Messiah4 points  

Take care gwern it was nice moderating in this sub reddit with you in the past.

BinaryResult4 points  

You're a legend /u/Gwern !!! Where do you think you will re-focus your attention now?

gwerforpresiedent3 points  

thanks Gwern. Good luck on your next projects.

Also, I'm interested in this material but am not sure if it is really accessible to me. Will I be able to understand the files and material if I've only got a BASIC understanding of webscraping/python?

gwern5 points  

If you understand HTML, then you can understand most of the files. Proper parsing and analysis of full datasets might require more Python skills than you have but at least it makes a good little hobbyist project to learn a new tool?

gwerforpresiedent2 points  

Thanks you. You have built a recognized and reputable name.

Gwern for President! 4 More Years!

[deleted]3 points  

[deleted]

scoopydoopypoopy3 points  

Thank you for all your effort gwern! It must have required some serious personal willpower to keep up with it all, and it is immensely appreciated

select1on3 points  

I am seeding this Torrent until dnstats.net dies.. so for a while I hope.

peacockvapor2 points  

You will be missed and thanks for your efforts. I have spent hours on your site. Read some stuff twice....Enjoy your next efforts.

I think a good send off would be for everyone with leftover change in there wallet donate it to Gwern!

SlimyGreenShit2 points  

I will miss you and I hope you still browse and give good information when you can. I hope they don't take down this subreddit, either, tomorrow. Either way, thank you for everything you've done, I'll catch you on the flip side.

Axaq2 points  

Fantastic work Gwern, this will help tremendously with my own ventures, good luck for the future pal.

BabyBunt2 points  

Thank you Gwern, for exhibiting a rarefied and luminescent passion that shown its inspirational light on so many others than just myself.

For anyone interested in listening to a text-to-speech version of the silk road forums, 37-half hour episodes are currently uploaded: https://youtu.be/IUdwmerw4yQ

ciphersexual2 points  

I love that you're still making these. Please don't stop until they're shown in The Met.

IGetDankShit1 point  

This is... interesting. I wish there was a way to search the audio for threads I want to listen to.

herodose19432 points  

Fuck that last line sent chills down my spine. It's sad to see you go Gwern but thank you for your amazing work.

ciphersexual1 point  

Did you google the last line too? I thought for sure it was "from something," but apparently it's just from Gwern.

herodose19431 point  

No I didn't, though it's no surprise that it's from Gwern.

syn1k12 points  

Unfortunately some or most of the blackbank market files I'm seeing just shows the login page. Either the script didn't log in first (or re-login) or blackbank probably had some sort of anti-DDOS mechanism to force signing in again.

gwern, thank you for all the incredible contributions over the years.

gwern2 points  

> Unfortunately some or most of the blackbank market files I'm seeing just shows the login page.

All at regular URLs, right? Yes, that happened with some markets. I tried to block as many of those as possible by disabling redirects and blacklisting login pages, but those BB pages are either from before I did that or were silent transclusions. (Some scrapers support running some sort of predicate on the download to check for strings consistent with error pages, to deal with ill-behaved sites which do not throw a HTTP header error like they are supposed to, but not wget.)

SecondWarning2 points  

What are your guys' thoughts on this?

https://www.reddit.com/r/DEAagents/

gwern2 points  

> Also visit: our friends at /r/buttcoin and /r/actualmoney.

Just some hater who isn't nearly as funny or acute as they think they are.

SelfishMe0072 points  

So are the archives a 50gb download or a terabyte?

gwern3 points  

Former.

SelfishMe0071 point  

So a Terabyte? And I'm super new to this site and the DNM's but the praise here to you is something extraordinary and from reviewing some of the contents of these archives you seem like a VERY smart guy. Wish I knew 1/2 of what you do. Thanks for the contribution and paving the way to.....endless possibilities?

VH-TJF1 point  

No not a 'terabyte'.
Because 'former' = 'the first one' = 50gb. He should have just answered '50gb' again for you... less keys to tap, less English for you to re-learn.

[deleted]1 point  

[deleted]

sobulbous1 point  

Have you been cast as a synth on Humans or something?

Seriously, this data is invaluable and your efforts always went above and beyond. Thank you.

[deleted]1 point  

[deleted]

gwern7 points  

I think it's safe. No one has reported back any time bombs so far.

PathlessDemon1 point  

I've only been subbed here to see just a handful of your helpful additions /u/Gwern, but thank you for everything you've done in this community.

I hope you're next project keeps you happy with a stellar fanbase as you've made here as a mod.

wootpedofundingijaid1 point  

Wow you even covered pedofunding? You should check out the newest form of it called pedo-co-operative where instead of paying for it you use your computer to mine darkcoins which are paid out to uploaders.

Markets are evolving, it just takes longer for some than others.

gwern1 point  

> Wow you even covered pedofunding? You should check out the newest form of it called pedo-co-operative where instead of paying for it you use your computer to mine darkcoins which are paid out to uploaders.

That sounds interesting. So how does that work, exactly? It seems like if you're doing is setting up a Darkcoin miner and sending block rewards off to some arbitrary address, it's the same as the crowdfunding site but far more of a hassle, and I wouldn't expect that to work. (Who wants to install or run a miner? That's pretty damning on its own.) So is there something more like an assurance contract or a Darkleaks-like spend-and-release-decryption-key mechanism?

wootpedofundingijaid1 point  

TBH i dont get it myself, i think you just mine for coins it goes into admins wallet and he decides how its distributed. Its kinda stupid idea, i liked the crowdfunding model better, less risky, more choice.

gwern1 point  

That is pretty dumb. Why don't the CP guys just use [Darkleaks](https://github.com/darkwallet/darkleaks)? This is the *perfect* use case: you want to monetarily incentivize the release of incremental content among mutually untrusting people online. Which is exactly what Darkleaks does.

(I know, I know - they're all too dumb/lazy/technically-unsophisticated to use Darkleaks. Leave me my cryptopunk dreams!)

r0k1t1 point  

Hey Gwern,
i think I'm not only speaking for myself, when I say that it truly has been an amazing time, since I get to know you earlier this year.
I cannot think of much people or things that have brought so much entertainment and interesting content to me.

While I'm also a bit sad to hear that you are stepping down as a mod and probably will spend less time on DNM research,
I also hope you will still hang out in IRC and/or reddit and we get to witness more of your amazement!

I doubt that I will ever get to know a person as unique and clever as you are :)

On the other hand, I was asking myself for some time now, where you take the energy and motivation from to do all the stuff you do. Espescially with (from what I could see) not that much love coming back from the community.
I'm following the whole DNM/SR Trial/etc. since about end '14 and a small oversaturation has set in on me too.

I hope you will find something as interesting as the DNM scene was and we get to know what its about.

If everybody would be honest, he/she would donate you a bit, for all your work and effort.
I have payed much more for things that had much less value than what you do.

I wish You all the best!

Greetings

r0kit

LedLevee1 point  

You mention that the SR1 model is surprisingly durable and that there seems little interest in developing multi-sig and trustless markets. I assume you refer to P2P and OpenBazaar like projects?

Why do you think that is? In my head nobody wants to really develop those because you can't make money off of them right? A trustless market has no commissions, correct?

If this is not true, could you explain a valid business model with a trustless market that also generates income?

Thanks for all that you've done for the community. Losing a huge asset today. Stay safe!

gwern3 points  

I think that a decentralized market doesn't necessarily need a lot of developer time or effort once it's implemented and so a funding model might not be important. It could go off donations or bounties. (Working on Bitcoin core isn't directly profitable either.) And since a distributed market could be effectively immortal, once it happens, it doesn't need to happen again, while centralized markets must be coded over and over with each new market.

Under some models, you might be able to turn a buck. Like in a multisig-style model with paid arbitrators, there might be a 'default arbitrator list', which would be profitable to be on since buyers are lazy.

ShulginsCat1 point  

> you might be able to turn a buck.

It would be great if we could figure out a model where the *developers* make a direct profit, rather than donations. This will give them an incentive to patch holes, provide upgrades, etc. Bittorrent has been a spectacular success for enabling illegal activity, but at some point it also became profit-oriented (TPB and uTorrent posting ads, and eventually BT itself going corporate). Maybe the same idea is feasible for P2P markets?

galaxyandspace1 point  

Thank you for all the info you have provided. Your reasons for stepping down sound quite similar to mine.

Good luck on all future endeavors.

troubledteendwight1 point  

Thanks for all your hard work! Awesome stuff here.

gramsadmin1 point  

Thank You for all your hard work

MXEMan1 point  

A deep and hearty thank you, gwern, for your efforts on behalf of all of us.

I feel much the same as you. I fear the lack of adaptation in the markets to obvious threats from thieves and LE will ultimately result in their demise. My personal business model moving forward is to gather a small, elite customer base and possibly work as a direct supplier to a dnm vendor or two.

DrinkMoreCodeMore1 point  

How many government special agents have contacted you since the subpoena ?

gwern1 point  

None.

DrinkMoreCodeMore1 point  

Dont you find that odd? You are def being monitored and your public/private reddit activity being logged at least by a few different government agencies.

gwern2 points  

I do find it a bit odd, but not too much, since I really am what I appear: a researcher who has very occasionally bought on the DNMs. If they are snooping through my Gmail and other accounts, as is sadly likely and why I encourage everyone to use my PGP key if they want to talk about anything sensitive, that ordinary reality is exactly what they will find (if they really did open up my China package, they will have been disappointed to find it really was just a pair of Bluetooth earphones), and they certainly won't find anything suspicious like me talking about running my DNM with baronsyntax.

corwin061 point  

> I give you back your memories.

This is what "information wants to be free" means. It means never having to forget anything ever again.

This is the history of the Internet.

Farfetchedchild1 point  

Thank you for all of your work. You've been a huge asset to not only this community, but to representing research and journalism (as it should be) as a whole.

Stellar_Surfer1 point  

AMAZING!

sapiophile0 points  

<3

Vendor_BBMC0 points  

Nobody bothered to document the early days of rock & roll, or hip hop. They just seemed like fads at the time.

You had the vision to know that all the chaos and flailing around in the darkness of the darknet would turn out to be important, important enough to document.

Elvis is dead (well, incarcerated), and even you are starting to think the best of the DNM fad might be over.

Its not. Its just a bit like 1962. It seems like all the markets are bad guys, the vendors are bad guys, and even LE are corrupt. Ross may have been the first good man, uncorrupted by the bitcoin, to have a vision for the future. Im sure he wont be the last, so dont let a bout of tor fatigue get you down.

The same anonymity which allows us to trade also allows us to rip each other off, so what's the point in working so hard when its all going nowhere?

It IS going somewhere. Every problem will be solved, so don't mourn the loss of a technological red-herring like fancy escrows.

Scammers are weak, and they are stupid. They dont believe their own rational mind that the best way of doing things always wins. Reputation on the darknet IS THE ONLY THING WORTH COVETING. Why throw it away for a few bitcoin, or because it doesnt seem to be moving forward? Because it will rival the GDP of major countries.

I appreciate you,Gwerningham. I can probably infer that youve been taking shit behind the scenes.

DNMs are going to be bigger, better and more important than even you thought.

Don't retire now, or you will miss the Beatles on Ed Sullivan.

Use of this site constitutes acceptance of our User Agreement and Privacy Policy. © reddit inc. All rights reserved.

REDDIT and the ALIEN Logo are registered trademarks of reddit inc.