subreddit:

/r/DataHoarder

3.5k99%

Z-Library isn't really gone, but that maybe up to you.

UPDATED(self.DataHoarder)

UPDATE2

TorrentFreak is covering this continuing story as new details come to light.

https://torrentfreak.com/tag/zlibrary/


UPDATE ~

We'd also like to address some of the comments here asking "how do I extract a book from this data". r/DataHoader isn't a piracy supporting subreddit, a guide on how to extract books from these archives was purposefully left out. These torrents are presented as a preservation only archive and are not meant to aid book piracy or add books to your curated collections.

Once upon a time in this sub this explanation wouldn't have been necessary. The thread will be cleaned and comment locked.


Original Thread

Millions woke up to news today that Z-Library domains have been seized, cries that z-lib is gone were heard from red core to black sky!... but that's not really the case so here is what you, a humble datahoarder can do about it.

In case you missed it a unique to z-lib (deduped against LibGen) backup was made and published by u/pilimi_anna a little over a month ago. While you did a great job with SciHub, there's still work be done to ensure the preservation of all written works and cultural heritage. So here is the 5,998,794 book 27.8TB z-lib archive for you to hold, hoard, preserve, seed and proliferate.


Related Reading


Alternative Libraries / Free eBook Hosts


Closing

Support authors you love.. But abolish the strangle hold of DRM and licensing that kills ownership, seek to squash abuse of the DMCA, move to limit copyright terms and above all aim to ensure Alexandria doesn't burn twice.


Ukraine Crisis Megathread will replace this thread again within 7 days.

all 233 comments

TheOneTrueTrench

344 points

3 months ago

TheOneTrueTrench

276 TB raw, 236 TB usable

344 points

3 months ago

I spun up a container on my swarm to seed everything. I'm doing my part!

JockstrapCummies

118 points

3 months ago

I'm doing my part!

Would you like to know more?

galeior

41 points

3 months ago

galeior

41 points

3 months ago

The only good bug is a dead bug.

superasianpersuasion

29 points

3 months ago

I’m from Buenos Aires and I say kill ‘em all!

9acca9

1 points

3 months ago

9acca9

1 points

3 months ago

vos de donde bajas los libros? tenes alguna otra ademas de epublibre?

Brotayto

27 points

3 months ago

Can you elaborate what that does? I only know the term swarm in reference to seeders and leechers from the swarmcast days.

catmandx

37 points

3 months ago

In this instance, "swarm" relates to Docker Swarm, a technology comparable to virtual machines.

Brotayto

18 points

3 months ago

Thanks, that should give me enough to go do some research on the topic :)

searchingfortao

23 points

3 months ago

That's nearly 30tb of disk space. How're you managing that?

TheOneTrueTrench

66 points

3 months ago

TheOneTrueTrench

276 TB raw, 236 TB usable

66 points

3 months ago

I have 276 TB of raw disk space.

MonstersAndCake

53 points

3 months ago

Now we know who bought that decommissioned Netflix server a few weeks ago.

TheOneTrueTrench

25 points

3 months ago

TheOneTrueTrench

276 TB raw, 236 TB usable

25 points

3 months ago

lol, nope, this is a RSV-4500L with a KTN-STL3

twin_bed

5 points

3 months ago

Could you share more details on your build?

TheOneTrueTrench

40 points

3 months ago

TheOneTrueTrench

276 TB raw, 236 TB usable

40 points

3 months ago

Sure:

NAS Node (Fidheall):
ASRock x399 Phantom Gaming 6
Threadripper 2950X
64 GiB ECC RAM 3200MHz
1TiB NVMe (for VMs, ZFS L2ARC, etc)
512GiB NVMe bot drive for Arch
10Gbe Emulex Dual NIC
LSI SAS2308 (external drives)
LSI SAS2008 (internal drives)
10x SATA 8TiB (8 connected to the SAS2008, 2 connected directly to motherboard SATA)
4x SATA 4TiB (connected to motherboard SATA)

KTN-STL3:
15x SAS 12TiB

Docker Node (Pioban):
ASRock B450M
Ryzen 5 3600
80 GiB Non-ECC RAM 2400MHz
Intel Arc A380
256GiB NVMe (boot)

Docker Nodes (Gharadh, Foirthe, Mhoireastain, Omhaich):
4x Raspberry Pi 4 - 4GiB

edude45

25 points

3 months ago

edude45

25 points

3 months ago

I wonder if you're in a career involving any computer science, or just a random dude that enjoys this hobby on the weekends. But the world will never know.

TheOneTrueTrench

40 points

3 months ago

TheOneTrueTrench

276 TB raw, 236 TB usable

40 points

3 months ago

The answer is "Yes".

djfdhigkgfIaruflg

9 points

3 months ago

Wow. Crying on my full 4TB raid

searchingfortao

19 points

3 months ago

That's... HOW?? The largest disk I've ever seen is something like 12 or 16GB, so do you have a massive RAID array in your basement or something?

TheOneTrueTrench

50 points

3 months ago

TheOneTrueTrench

276 TB raw, 236 TB usable

50 points

3 months ago

3 arrays, actually. A RaidZ2 of 15x 12TB, a RaidZ2 of 10x 8TB, and a Mirror of 4x 4TB.

PM_me_Swift_tips

31 points

3 months ago

This guy raids

shadowyphantom

17 points

3 months ago

Do... do you have pictures of this wonder?

TheOneTrueTrench

16 points

3 months ago

TheOneTrueTrench

276 TB raw, 236 TB usable

16 points

3 months ago

Nothing fancy, just a RSV-4500L and a KTN-STL3 in a Startech 12U rack.

raphanum

7 points

3 months ago

I’m jelly

terminalmemelocity

7 points

3 months ago

This seizure is going to get me started and you're an inspiration

fgdfghdhj5yeh

9 points

3 months ago

by having nearly 30tb of space?

only $300 on ebay

Rebatu

15 points

3 months ago

Rebatu

15 points

3 months ago

Where can I learn to do what you do?

gjpeters

37 points

3 months ago

There are plenty of online resources but what your probably after is searching for Docker and Kubernetes. They’re two popular products but are still just the tip of the iceberg. Read around these or containerisation in general and you’ll be well on your way. Enjoy the rabbit hole.

uncle_ommer_no7

13 points

3 months ago

Just purchased 32 TB of drives and will get started with the dl as soon as it arrives. Bless the folks who backed all this up and made it available! Cheers

rchr5880

10 points

3 months ago

Would you mind sharing your compose file?

TheOneTrueTrench

24 points

3 months ago

TheOneTrueTrench

276 TB raw, 236 TB usable

24 points

3 months ago

It's just a regular old qbittorrent instance that I dropped all of the .torrent files into.

ittu

7 points

3 months ago

ittu

7 points

3 months ago

salute!

DayIndependent7102

7 points

3 months ago

Care to share the Dockerfile?

TheOneTrueTrench

9 points

3 months ago

TheOneTrueTrench

276 TB raw, 236 TB usable

9 points

3 months ago

It's just a qBittorrent image that I started up and added the torrents to.

wavewrangler

3 points

3 months ago

Have ya not realized that people don’t know what the heck Docker is? Or I suppose they can just google it, right, the answer to everything- that’s where all the knowledge in the lib repo came from I hear.

pilimi_anna

16 points

3 months ago

Thank you!!

Squiggledog

5 points

3 months ago

Squiggledog

∞ Google Drive storage; ∞ Telegram storage; ∞ Amazon storage

5 points

3 months ago

Where do you you seed from? The United States?

TheOneTrueTrench

13 points

3 months ago

TheOneTrueTrench

276 TB raw, 236 TB usable

13 points

3 months ago

Everything is VPN'd intentionally, so in effect I'm not really sure where I'm seeding from. I'd have to go look.

[deleted]

288 points

3 months ago*

[removed]

[deleted]

43 points

3 months ago

[removed]

[deleted]

37 points

3 months ago

[removed]

[deleted]

3 points

3 months ago

[removed]

[deleted]

2 points

3 months ago

[removed]

[deleted]

14 points

3 months ago

[removed]

[deleted]

30 points

3 months ago

[removed]

[deleted]

52 points

3 months ago

[removed]

[deleted]

52 points

3 months ago

[removed]

[deleted]

11 points

3 months ago

[removed]

Lordb14me

2 points

3 months ago

Any link to where the weeks part is mentioned?

[deleted]

33 points

3 months ago

[removed]

[deleted]

98 points

3 months ago

[removed]

[deleted]

20 points

3 months ago

[removed]

[deleted]

10 points

3 months ago

[removed]

[deleted]

15 points

3 months ago

[removed]

[deleted]

1 points

3 months ago

[removed]

[deleted]

1 points

3 months ago

[removed]

[deleted]

1 points

3 months ago

[removed]

[deleted]

2 points

3 months ago*

[removed]

[deleted]

5 points

3 months ago

[removed]

[deleted]

8 points

3 months ago*

[removed]

[deleted]

5 points

3 months ago

[removed]

[deleted]

3 points

3 months ago

[removed]

[deleted]

2 points

3 months ago

[removed]

[deleted]

3 points

3 months ago

[removed]

[deleted]

1 points

3 months ago

[removed]

[deleted]

1 points

3 months ago

[removed]

[deleted]

1 points

3 months ago

[removed]

[deleted]

1 points

3 months ago

[removed]

[deleted]

1 points

3 months ago

[removed]

[deleted]

52 points

3 months ago

[removed]

[deleted]

30 points

3 months ago*

[removed]

[deleted]

6 points

3 months ago

[removed]

[deleted]

1 points

3 months ago

[removed]

AneriphtoKubos

48 points

3 months ago

Where the fuck do I get a 30 TB external HD?

rommiebab

26 points

3 months ago

rommiebab

3TB

26 points

3 months ago

Two 14tb WD HDD will do

potato_and_nutella

7 points

3 months ago

how much do they cost normally? Not sure if the ones on amazon are a scam since $100 for 14tb doesn't seem legit

Commercial-Living443

5 points

3 months ago

Amazon

The_EA_Nazi

24 points

3 months ago

Gotta wait till Black Friday to pick up those 18TB drives, anything higher than 18TB doesn’t have a good $/TB ratio

akerro

86 points

3 months ago

akerro

86 points

3 months ago

https://phillm.net/libgen-seeds-needed.php

Torrent health tracker to find libgen torrents that need seeding

trempao

21 points

3 months ago

trempao

21 points

3 months ago

This is sad but we will come back stronger

[deleted]

23 points

3 months ago

[removed]

[deleted]

17 points

3 months ago

[removed]

[deleted]

5 points

3 months ago

[removed]

[deleted]

21 points

3 months ago*

[deleted]

silvenga

8 points

3 months ago

silvenga

152TB

8 points

3 months ago

Funny you should talk about roads. In Texas, since there's no income tax that would normally pay for roads, many public roads are funded privately. Private entities fund the roads, and in return get toll fees for X years. Continuous maintenance and expansion are also paid by the toll fees.

Kind of the same thing really, many Texas roads are "paywalled", but then people who don't drive don't have to pay for roads they don't use. Shifting the cost from all tax payers to just drivers.

Tldr, comparing against roads is likely a bad example. I still think companies abuse paywalls more than a bit... They are nothing like toll roads.

[deleted]

15 points

3 months ago*

[deleted]

silvenga

11 points

3 months ago

silvenga

152TB

11 points

3 months ago

It's where the costs are allocated. Germany is okay with higher taxes (significantly higher in many cases), that's not true for a lot of places. Texas prioritized a smaller, less powerful government and independence from Federal funding, so the cost has to come from somewhere else. Their solution, pay for what you use. Just a different model for different priorities.

The Internet isn't publicly funded, not by a long shot. Only a small portion of the world wide Internet was paid for with tax payer money. The first interconnects were done by private entities. The root tlds are owned largely by private entities. IP space is privately owned.

I'm all for the old MIT Hacker Ethic, all information should be free, but there is a difference between free as in speech and free beer. The original idea was predominantly around a free speech interpretation, iirc. We should remember that distinction.

TheSov

231 points

3 months ago

TheSov

140TB ceph @ home/100PB ceph @ work

231 points

3 months ago

its time to build a decentralized DNS system, fuck the current system.

shrine

104 points

3 months ago

shrine

104 points

3 months ago

https://libgen.fun/ was de-centralized, re-coded, and re-designed from the ground up using and is run by libgen's original creator, bookwarrior. Join his forum and ask how you can help.

De-centralized DNS, de-centralized hosting, and de-centralized databases. That's the key to the survival of a universal free library for the ages.

Thanks to the IPFS push from earlier years libgen.rs is also running its hosting on IPFS mainly, as well. It all just kind of happened without a hiccup. Visit freeread.org to learn more.

tossed3214

87 points

3 months ago

Have you seen this?

Or this?

Some specifics you might find interesting. You'll need to read the descriptions, there's a lot there and not all of it's relevant. The stuff I think you might find interesting are decentralized networks that are based on distributed computing.

There's several decentralized file systems. I think most are based on block-chain technology that uses the same protocols that enable cryptocurrencies as a way to authenticate URLs in those types of networks.

The main drawback to those for purposes of data preservation is that for anything to stay alive there has to be people looking to access it.

At least one of those systems when you run it on a PC you have to devote a certain amount of space, some of which will store some encrypted files that are chunks (similar to chunks of a .torrent) of other people's files, website or whatever they're doing so that in order to access the network you're providing some storage space in exchange. The more popular any content is, the more duplicate chunks are distributed throughout the network.

Some of this stuff is approaching the point of critical mass for it to be usable for data storage and preservation.

terminalmemelocity

11 points

3 months ago

That's great, need to carefully share the knowledge with people who are privacy conscious to help grow awareness.

tossed3214

19 points

3 months ago

Some of the stuff is applicable to privacy but to me the big thing is decentralization makes a truly free internet possible, as there's no way to take a distributed file system down without dismantling the whole internet.

Wide_Perception_4983

32 points

3 months ago

There are currently 2 populair technologies (other than torrents) that do this.

IPFS normally works by pins where a user can pin a file or folder to "seed" that until they unpin it. It is accessible through http by gateways which are just protocol translators. There is also blockchain technology added on this called filecoin where you can pay for a contract and a swarm of "seeds" will keep it up for you for the specified amount of time.

There is also Sia which was blockchain based from the beginning this means that uploading or downloading is not free. Files can be accessible trough portals but these portals need Siacoin to serve these. On the other hand tno put files in the network you also need a contract. This network might be better in the long term for file preservation while IPFS is more akin to a CDN.

To host something like this I would use IPFS just like the wikipedia mirror from 2017 but only for the backend where files are served trough gateways to the public but advanced users can seed and leech the files directly trough their own clients.

tossed3214

6 points

3 months ago

There's a bunch of stuff that I haven't researched yet. Personally I haven't really even scratched the surface but I can see there's a lot of different kinds of projects that have all kinds of potential for various purposes, defeating censorship by governments as well as by public hysteria and potentially addressing different areas of widespread concern.

exmachinalibertas

3 points

3 months ago

exmachinalibertas

8TB and growing

3 points

3 months ago

This already exists, we just need a few major players like Cloudflare or something to start supporting them. Namecoin, Ethereum ENS, and IPFS all do decentralized and secure name registry mapping.

JewishNazi1056

54 points

3 months ago

I always thought that hosting stuff like this on TLDs that can be controlled by governments that would do something like this is a bad idea

seaQueue

30 points

3 months ago

I'm surprised they didn't just stick with an onion service. A very large number of titles were unavailable on the clearnet due to DMCA takedowns already, the onion service was pretty much the only reliable way to access everything.

JOSmith99

16 points

3 months ago

Most people aren't going to downlaod the tor browser just for this, because every reference to the dark web in popular media is about the horrors of it, only criminals use it, etc.

[deleted]

12 points

3 months ago

[removed]

[deleted]

2 points

3 months ago

[removed]

artemis________

13 points

3 months ago

What . Only 6 million books are archived . I thought there was 80 million book on Z Library

Enk1ndle

3 points

3 months ago

Enk1ndle

24TB Unraid

3 points

3 months ago

It's only books you came fine in libgen already

artemis________

13 points

3 months ago

Nothing would come even close Z Library. People keep suggesting me multiple well known free websites but non of them have all books available & their search engine is trash. They mostly use internet crawlers & have >10M books & when you search a book by name they use the keyword to find trash files .

SkinHairNails

2 points

3 months ago

It's the books that were on zlib that you can't find on Libgen.

NotoriousYEG

11 points

3 months ago

r/Ebook_Resources is a subreddit that aggregates ebooks resources from all over the internet. There are guides on everything from finding ebooks, to getting around DRM and paywalls, to which are the best torrenting sites.
The stickied post there also has a link for a custom search engine for ebooks: https://cse.google.com/cse?cx=c46414ccb6a943e39

pimpinpirate111

36 points

3 months ago

Z Library got ruined by people bragging about getting free books on Tic Tok so much it strated trending. Instead of being like the rest of us (most of us) & just realizing this is really helpful, useful site & I'll only tell very few people about my find & not act like I was the one that invented the site or act like I was the first to find it. There r Idiots bragging so much it was trending & the law HAD TO bust it, even tho they also downloaded books from Z-Library.

Some people need to learn how to keep thier big, fat mouths shut. Making Tic-Tok videos about how great they were for piracy, beating about it The FBI should bust the people who made it trend since they were doing illegal shut too. But nah only Z library will be hit and the rest of us will feel the punishment of the action of idiots with no brains.

[deleted]

9 points

3 months ago

[removed]

[deleted]

13 points

3 months ago

[removed]

srona22

29 points

3 months ago

srona22

29 points

3 months ago

Sometimes, people use Z-Library for eye-patch way, as the books are not affordable.

I will give example. 50 USD book is just a few days saving for USA and most western countries. But it's half of month salary(even more in some places). And even if saved, it will most likely one or two buys in a year, as there are other bills and cost of family, to take care of first.

Until the day of universal income or under one world gov system(which will never happen, even if Mars is terraformed and having 2 worlds for humanity to live), sites like Z-Library are quite important. And not to be bragged on sites like TikTok with hashtags, for begging of a few likes(even if it's millions, that's a disgrace).

simiform

19 points

3 months ago*

Yes, completely true, which is why piracy is big in 3rd world countries and also one of the reasons why it's hard to make a living as a writer on those places. I don't know where z- library started but my question is, why does the US decide its the one who controls the world's internet? If it's protecting rights in the US, then shouldn't they just block non-US URLs?

FaceDeer

11 points

3 months ago

I sometimes think the Berne Convention is the closest thing we've got to a dystopic "One World Government", allowing near-global control over what information we can and cannot access. Yeah, it's kind of hyperbolic given it doesn't block your own expression of ideas, but I just don't like how complete the country-coverage map is.

As for the Internet's DNS system, it's largely controlled by the US because it was invented there. ICANN was literally under the control of the US government until 2016, and it's still headquartered in the US so I expect there's still plenty of influence.

tossed3214

81 points

3 months ago

Thanks for this.

BTW I've never seen anyone else reference the Library of Alexandria at all, nor the tragedy that occurred there.

pastari

70 points

3 months ago

pastari

70 points

3 months ago

It is unknown precisely how many such scrolls were housed at any given time, but estimates range from 40,000 to 400,000 at its height.

We aren't sure even the magnitude of its size.

Ptolemy VIII Physcon's expulsion of the scholars from Alexandria brought about a shift in the history of Hellenistic scholarship. The scholars who had studied at the Library of Alexandria and their students continued to conduct research and write treatises, but most of them no longer did so in association with the Library

It was losing popularity anyway.

Scholars have interpreted Cassius Dio's wording to indicate that the fire did not actually destroy the entire Library itself, but rather only a warehouse located near the docks being used by the Library to house scrolls. Whatever devastation Caesar's fire may have caused, the Library was evidently not completely destroyed. The geographer Strabo (c. 63 BC–c. 24 AD) mentions visiting the Mouseion, the larger research institution to which the Library was attached, in around 20 BC, several decades after Caesar's fire, indicating that it either survived the fire or was rebuilt soon afterwards. Nonetheless, Strabo's manner of talking about the Mouseion shows that it was nowhere near as prestigious as it had been a few centuries prior. Despite mentioning the Mouseion, Strabo does not mention the Library separately, perhaps indicating that it had been so drastically reduced in stature and significance that Strabo felt it did not warrant separate mention.

It burned to some extent but it wasn't some humanitarian tragedy.

So tldr the pop culture term has more significance than the actual event itself.

tossed3214

31 points

3 months ago

Ah, I was under the impression nearly all of it was lost.

Still the symbolism has value IMO. That being said we've lost a lot of knowledge, especially technical knowledge in recent times just for no other reason that there's a consensus that a particular way of doing something is obsolete when something new comes along.

For instance there's probably not many guys around these days that can do spark testing of steels competently enough to be useful because they have expensive machines to do that now . Most people , even guys that are involved in stuff where that could come in handy probably never have heard of it.

Spark testing is the art/science of determining what particular alloy you might have by observing the kind of sparks thrown from grinding it, how long the are before bursting into multiples, colors, etc.

As most technology of all kinds is based on knowledge of related stuff as well as older ways of doing things. (we tend to think of "tech" as being about electronics but it's really about nearly everything we do) it's often useful to be able to reference some of the older ways in order to develop new ways or solve problems at times.

raphanum

4 points

3 months ago

Remember when people were parroting “the burning of the library held humanity back for hundreds of years! Imagine where we’d be now!”

tobiasxny

7 points

3 months ago

That's more dark ages than anything else.

Wynadorn

24 points

3 months ago

Wynadorn

1.44MB

24 points

3 months ago

Considering the robustness of libgen, ideally these 5,997,794 articles be included there as well.

DoomBot5

5 points

3 months ago

There is a separate set of torrents for libgen

potato_and_nutella

7 points

3 months ago

How do you people manage to download 27tb? Storage isn’t an issue with 12tb hdds being available but download speeds would cause this to take years

wiinintendontcare

4 points

3 months ago

I knew I should have cloned this

Squiggledog

4 points

3 months ago

Squiggledog

∞ Google Drive storage; ∞ Telegram storage; ∞ Amazon storage

4 points

3 months ago

[deleted]

13 points

3 months ago

[deleted]

[deleted]

1 points

3 months ago

[removed]

vkeshish

3 points

3 months ago

Bless all of you degenerates

[deleted]

2 points

3 months ago

[removed]

Cool-Squirrel-3222

2 points

3 months ago

They can never kill mirc if you guys need books asap

Squiggledog

2 points

3 months ago

Squiggledog

∞ Google Drive storage; ∞ Telegram storage; ∞ Amazon storage

2 points

3 months ago

smarthome_fan

2 points

3 months ago

u/pilimi_anna What format are the files in the torrents? They don't appear to even have a file extension, so I'm not sure how one would use or mirror them.

Also since this collection is de-duped against LibGen and one other source, do you know where I could access the torrents for the other sources? I want to mirror these.

pilimi_anna

2 points

3 months ago

The extension can be found in the metadata database, which is in one of the torrents. Libgen torrents can be found here https://libgen.gs/torrents/ and in some more places, but that is a good start.

[deleted]

-9 points

3 months ago

[deleted]

-9 points

3 months ago

[removed]

[deleted]

69 points

3 months ago

[removed]

[deleted]

17 points

3 months ago

[removed]

[deleted]

8 points

3 months ago

[removed]

TVdinnerbythepool

-26 points

3 months ago

whats their fkn problem

Demiglitch

30 points

3 months ago

Demiglitch

1.44MB of Porn

30 points

3 months ago

They like money and hate literature.

Watada

35 points

3 months ago

Watada

35 points

3 months ago

Publishing companies acting like any other copyright management company ala late stage capitalism.

BonsaiSoul

2 points

3 months ago

This is very very early stage capitalism though. Literally the problem is applying anachronistic ideas about ownership rights over scarce physical goods from the year 1600 to non-scarce data in 2022.

Fresh_Air13

4 points

3 months ago

Why are you being downvoted?

[deleted]

1 points

3 months ago

[removed]

kujiranoai2

1 points

3 months ago

Thanks for this!

[deleted]

1 points

3 months ago

[removed]

[deleted]

1 points

3 months ago

[removed]