Failure to Archive RI, Feb 2019: My Surrender

Moderators: Elvis, DrVolin, Jeff

Failure to Archive RI, Feb 2019: My Surrender

Postby JackRiddler » Wed Feb 13, 2019 4:30 pm

.

Since late last year it has been my serious plan to create a new archive of the entire board, much like the one provided by Joao back in November 2016.
viewtopic.php?f=33&t=40198

(Also, to spend less time here after providing that.)

Sorry everyone, I don't know what I've been doing wrong. Here's the report of my failure:

Using two different versions of A1WD by Microsys, both the most current as well as the one used by Joao in 2016 (7.7.0), I made several major efforts. But they all failed.

The first couple of attempts ran in the background for several days in Dec and Jan. I made the insane attempt to download the site along with all of the displayed external graphics as resident. This was stupid! It was never going to work. I stopped trying that.

But even without it, it became clear I'd have to wait until I had access to faster Internet.

Once I had a faster connection, I tried again several times, using various settings offered by the program.

On the most recent try, I let the scan run for more than a week. It still wasn't done when I finally stopped it. The stored files had reached 18GB and more than 300,000 thread pages! This is wrong, given the board's current total of 39,000 threads in total and 646,000 individual posts, with no more than 15 posts per page. There might be about 100K thread pages, no more. In Nov 2016 the site had 36,000 threads and 82,000 pages. Two years ago, Joao's archive was under 4GB, just 1.2GB as a zip file.

Anyway, despite this awesome size, the results lacked a great many files that can be found on the site and especially lacked many index pages, unlike Joao's perfect archive of 2016. But there were also countless other identical pages in triplicate, quadruplicate, or ten times. Which is how it turned into 18 GB and was still growing.

So whatever I was doing, or whatever the program was doing, it was a fuck up and I have no idea why. I suppose it's also possible that it's neither me nor the program, but that the problem is caused by some change in the board settings (to do with robots or some such).

At this point I must surrender. This was a lot of work for nothing. (The program may run in the background, but of course one constantly checks it and has a slower machine because of it.)

Regulars, I invite any of you to try, maybe with another downloader, or with superior skills and experience, and see if you can manage it.

It seemed so easy!

.
We meet at the borders of our being, we dream something of each others reality. - Harvey of R.I.

To Justice my maker from on high did incline:
I am by virtue of its might divine,
The highest Wisdom and the first Love.

TopSecret WallSt. Iraq & more
User avatar
JackRiddler
 
Posts: 16007
Joined: Wed Jan 02, 2008 2:59 pm
Location: New York City
Blog: View Blog (0)

Re: Failure to Archive RI, Feb 2019: My Surrender

Postby Grizzly » Thu Feb 14, 2019 12:57 am

Well, crap thanks for trying Jack. Interestingly enough, I'm finding this problem across many boards I frequent. Lost archives of documented history and crimes thereof. Reminds me of the burning of Alexandria to be honest. But, hey! Lets not be so pessimistic. This ought to cheer everyone up!



Image

Sing with me, yall!
“The more we do to you, the less you seem to believe we are doing it.”

― Joseph mengele
User avatar
Grizzly
 
Posts: 4908
Joined: Wed Oct 26, 2011 4:15 pm
Blog: View Blog (0)

Re: Failure to Archive RI, Feb 2019: My Surrender

Postby Harvey » Thu Feb 14, 2019 6:48 am

HTTrack. Free. It works. (Usually)

https://www.httrack.com/page/2/
And while we spoke of many things, fools and kings
This he said to me
"The greatest thing
You'll ever learn
Is just to love
And be loved
In return"


Eden Ahbez
User avatar
Harvey
 
Posts: 4200
Joined: Mon May 09, 2011 4:49 am
Blog: View Blog (20)

Re: Failure to Archive RI, Feb 2019: My Surrender

Postby JackRiddler » Thu Feb 14, 2019 7:04 am

.

If no one else does it, I will try it again sooner or later, with other software. Just not right away. ;-)

.
We meet at the borders of our being, we dream something of each others reality. - Harvey of R.I.

To Justice my maker from on high did incline:
I am by virtue of its might divine,
The highest Wisdom and the first Love.

TopSecret WallSt. Iraq & more
User avatar
JackRiddler
 
Posts: 16007
Joined: Wed Jan 02, 2008 2:59 pm
Location: New York City
Blog: View Blog (0)

Re: Failure to Archive RI, Feb 2019: My Surrender

Postby JackRiddler » Thu Feb 14, 2019 7:56 am

.

Grizzly: Please let's not overstate this. So dramatic, man.

We are not the library of Alexandria! Please! We're not even the Lego version. In my opinion, RI counts as a deserving set of shelves in one of the reading rooms.

As for this failure so far, it is not due to "THEM." I'm copping to either incompetence, or ignorance, or a gaping lack of imagination coupled with stubborness. (Once I decide to do something a certain way, I just don't want to give it up, even past the point where it can be called stupid of me. By the way, doing the same thing over and over expecting a better result is a definition of stupidity, not insanity.) There's some damn way to do it right, probably a laughably easy one, and I will find it and do it, or someone else will.

The prospect of memory-holing the Internet is real. That is one reason we should archive, among several. But I see the opposite has so far obtained -- for the most part, censorship not withstanding. In the West and in most of the world, we're in the info overload scenario. Everything worth knowing is drowning in layers of new shit, produced daily at an exponential rate. Without a doubt, more info is accessible than ever in all categories (bad, good, real, fake, important, trivial, very old and all-too-recent). For the moment, it is still the case that almost nothing that has been published online has been lost, and almost everything can still be found. I'd say that the New Internet so far has sapped the will to learn, not the possibility.

.
We meet at the borders of our being, we dream something of each others reality. - Harvey of R.I.

To Justice my maker from on high did incline:
I am by virtue of its might divine,
The highest Wisdom and the first Love.

TopSecret WallSt. Iraq & more
User avatar
JackRiddler
 
Posts: 16007
Joined: Wed Jan 02, 2008 2:59 pm
Location: New York City
Blog: View Blog (0)

Re: Failure to Archive RI, Feb 2019: My Surrender

Postby Belligerent Savant » Thu Feb 14, 2019 11:07 am

.


I'll give it the 'ol college try once my schedule clears up in the coming week+, or thereabouts. Any success (or likely failure) will be shared here.
User avatar
Belligerent Savant
 
Posts: 5575
Joined: Mon Oct 05, 2009 11:58 pm
Location: North Atlantic.
Blog: View Blog (0)

Re: Failure to Archive RI, Feb 2019: My Surrender

Postby Grizzly » Thu Feb 14, 2019 12:37 pm

Dramatic? I thought It was humorous; gallows humour perhaps, but funny none the less. You know, cause "we are the baddies"...



And Alphabet is definitely a military project,. that very well could memory hole everything.
“The more we do to you, the less you seem to believe we are doing it.”

― Joseph mengele
User avatar
Grizzly
 
Posts: 4908
Joined: Wed Oct 26, 2011 4:15 pm
Blog: View Blog (0)

Re: Failure to Archive RI, Feb 2019: My Surrender

Postby JackRiddler » Thu Feb 14, 2019 1:41 pm

Oh, right, humanfunction:humor.
.functionfail:retry
.functionfail:retry
.functionfail:retry
.functionfail:retry
.functionfail:retry
.functionfail:retry
...
We meet at the borders of our being, we dream something of each others reality. - Harvey of R.I.

To Justice my maker from on high did incline:
I am by virtue of its might divine,
The highest Wisdom and the first Love.

TopSecret WallSt. Iraq & more
User avatar
JackRiddler
 
Posts: 16007
Joined: Wed Jan 02, 2008 2:59 pm
Location: New York City
Blog: View Blog (0)

Re: Failure to Archive RI, Feb 2019: My Surrender

Postby DrEvil » Thu Feb 14, 2019 2:33 pm



Yup. Tried it last night and it seemed to be working fine. Didn't get the whole forum (I forgot to uncheck something so I was downloading every image, both on the board and embedded from other sites), but the parts I downloaded worked just fine.
"I only read American. I want my fantasy pure." - Dave
User avatar
DrEvil
 
Posts: 4144
Joined: Mon Mar 22, 2010 1:37 pm
Blog: View Blog (0)

Re: Failure to Archive RI, Feb 2019: My Surrender

Postby JackRiddler » Thu Feb 14, 2019 2:58 pm

Yeah, that's just what I was thinking, when I first tested it. Then it went wrong... How can we persuade you to go ahead and do the whole thing?
We meet at the borders of our being, we dream something of each others reality. - Harvey of R.I.

To Justice my maker from on high did incline:
I am by virtue of its might divine,
The highest Wisdom and the first Love.

TopSecret WallSt. Iraq & more
User avatar
JackRiddler
 
Posts: 16007
Joined: Wed Jan 02, 2008 2:59 pm
Location: New York City
Blog: View Blog (0)

Re: Failure to Archive RI, Feb 2019: My Surrender

Postby JackRiddler » Thu Feb 14, 2019 3:51 pm

Grizzly, and here I am now thinking about actual libraries that have burned in recent years.

The Iraqi library and museum.
viewtopic.php?f=8&t=31490&p=666478&hilit=artificial+intelligence#p666478

And, just as horrific, and barely noted, the Brazilian national library last year.

Complete loss of analogue tape recordings of hundreds of now extinct indigenous languages.

All the sites destroyed by fundamentalists (and imperialists) in the forever wars...

.
We meet at the borders of our being, we dream something of each others reality. - Harvey of R.I.

To Justice my maker from on high did incline:
I am by virtue of its might divine,
The highest Wisdom and the first Love.

TopSecret WallSt. Iraq & more
User avatar
JackRiddler
 
Posts: 16007
Joined: Wed Jan 02, 2008 2:59 pm
Location: New York City
Blog: View Blog (0)

Re: Failure to Archive RI, Feb 2019: My Surrender

Postby MacCruiskeen » Thu Feb 14, 2019 6:11 pm

Jack, many thanks for your efforts.

sent from my fecken tablet, yo
"Ich kann gar nicht so viel fressen, wie ich kotzen möchte." - Max Liebermann,, Berlin, 1933

"Science is the belief in the ignorance of experts." - Richard Feynman, NYC, 1966

TESTDEMIC ➝ "CASE"DEMIC
User avatar
MacCruiskeen
 
Posts: 10558
Joined: Thu Nov 16, 2006 6:47 pm
Blog: View Blog (0)

Re: Failure to Archive RI, Feb 2019: My Surrender

Postby DrEvil » Thu Feb 14, 2019 7:06 pm

JackRiddler » Thu Feb 14, 2019 8:58 pm wrote:Yeah, that's just what I was thinking, when I first tested it. Then it went wrong... How can we persuade you to go ahead and do the whole thing?


I'll give it another go, just have to fiddle with some of the filters. No promises though, not until I have the whole forum on my hard drive in readable form.
"I only read American. I want my fantasy pure." - Dave
User avatar
DrEvil
 
Posts: 4144
Joined: Mon Mar 22, 2010 1:37 pm
Blog: View Blog (0)

Re: Failure to Archive RI, Feb 2019: My Surrender

Postby Grizzly » Fri Feb 15, 2019 12:18 am

Grizzly, and here I am now thinking about actual libraries that have burned in recent years.


Same shit in Aleppo.... destroyed libraries and museums, churches.... I'll try to find the Anthropological paper I read, on this very thing.
“The more we do to you, the less you seem to believe we are doing it.”

― Joseph mengele
User avatar
Grizzly
 
Posts: 4908
Joined: Wed Oct 26, 2011 4:15 pm
Blog: View Blog (0)

Re: Failure to Archive RI, Feb 2019: My Surrender

Postby Jerky » Sat Feb 16, 2019 5:05 am

Fucking Hell. That's ridiculously depressing when considered as a whole.



JackRiddler » 14 Feb 2019 19:51 wrote:Grizzly, and here I am now thinking about actual libraries that have burned in recent years.

The Iraqi library and museum.
viewtopic.php?f=8&t=31490&p=666478&hilit=artificial+intelligence#p666478

And, just as horrific, and barely noted, the Brazilian national library last year.

<b>Complete loss of analogue tape recordings of hundreds of now extinct indigenous languages.</b>

All the sites destroyed by fundamentalists (and imperialists) in the forever wars...

.
User avatar
Jerky
 
Posts: 2240
Joined: Fri Apr 22, 2005 6:28 pm
Location: Toronto, ON
Blog: View Blog (0)

Next

Return to General Discussion

Who is online

Users browsing this forum: No registered users and 150 guests