Failure to Archive RI, Feb 2019: My Surrender

Moderators: Elvis, DrVolin, Jeff

Re: Failure to Archive RI, Feb 2019: My Surrender

Postby JackRiddler » Mon Feb 18, 2019 8:22 am

.

Here, crossposted, is my last post until I succeed with the archive. This is not a departure, but an attempt to shift to other activities for a while. I like that I wrote this and do not consider it a waste of time. Hope you enjoy reading it. If you do, it goes well with my second-to-last post.

JackRiddler » Mon Feb 18, 2019 7:13 am wrote:
Sounder wrote:Many folk go to 'Capitalism' as the cause of so much bad stuff, yet the Soviets or Chinese never thought about ecological upheaval either.


Others, like you, prefer to deny that the Soviet regime pursued the same mode of production as their Western capitalist competitors. They were clunky and stupid and did so to an even greater ecological extreme, beginning in an era when ecology was barely a concern in either ideological camp, under conditions of permanent attack from the outside and with a very weak capital base. The social creeds and the societies differed, but the productionist organization of the Soviet societies' economic powers did not. Ostensibly they believed they were managing the "capitalist stage" with an eventual (fictional) horizon of communism in some far future after the Cold War. In their industrialization phase, party planners looked at the real-existing capitalism of the West and decided, correctly, that the industrial takeoff had been preceded by a phase of "primitive accumulation" requiring vast amounts of colonial forced labor and destruction of traditional sustenance economies. They decided to use the Slavic peasantry and the indigenes of Siberia and Central Asia as the internal colonies for the purpose of building the infrastructure and accumulating the capital for the industrial take off. They also felt surrounded and compelled to follow the same models of industrialism and macro-accounting that prevailed in the Western capitalist countries, including the pursuit of enterprise profit as an end in itself. They adopted the fateful GDP, a god that only knows always how to add and never to subtract. The regime was party dictatorship by open violence and the big-lie creed was "Communism," but the economic practice is properly characterized as state capitalism, a.k.a. "real-existing socialism." In their final decades, they did their damndest to reproduce the consumerist prosperity model of the West, and failed, miserably. By comparison, anyway: the "standard of living" was better than in the capitalist-controlled Third World. At least they helped Cuba escape the fates of the rest of the Caribbean and Central American regions, which are blessed with capitalist regimes. Rest in hell, motherfuckers.

China today is the most important industrial producer and on its way to becoming the new financial center of the global capitalist system. Outside its borders it is valorized by the Western "markets" for playing capitalism by the neoliberal rules, for understanding how business is supposed to work, for its productive and disciplined labor regime, and for its breakneck pursuit of technological dominance. Never mind the bodies, it is considered the biggest capitalist success story of recent decades. When the great lovers of private property like Bill Gates and Steven Pinker talk about the "decline of extreme poverty," they know at least 50% of that trend is Made in China. And yet they credit this to capitalism, not communism, and to China's commitment to meritocratic management of maximizing production integrated into the global capitalist markets. The oligarchic practices and corruption are no more extreme than elsewhere and China is on its way to producing the majority of the world's billionaires, if it has not already. They go to Davos too, when they're not being arrested in Canada. China demonstrates, to the barely hidden satisfaction of Western neoliberal ideologists, that democracy and capitalism need not go together, and are historically separable phenomena. The message from large segments of the Western capitalist ruling class has been that "we" need to be more like China -- or else! Trump does not differ in his commitment to the same neoliberal creed of how labor should behave. Choking in its smog, today China is showing about as much or slightly more concern for ecological sustainability as the Western regimes. It is engaged simultaneously in the biggest projects to convert to solar energy, and also opening enough new coal mines to provide its outsize share in overdetermining the planetary catastrophe. Both of these industries are run by the capitalist rules.

In writing the above, I do not expect to influence your false and confusionist use of terms and determination to stick to your ideological construct of "CC." Your benightenment is your own. I cannot keep you from bizarroland riffs about the Boston Marathon bombing as part of an operation designed to deliver a socialist climate-change brainwash. (Wow!) I write this in the hope anyone reading this exchange between us gets the damn point, and for use in future writing. (I think the above two paragraphs will serve that end as outlines.) I feel no need to differ with your critique of the way the corporate media present the ecological issues, and laugh at your attempt to associate my thinking with that babble.

Your attempt to wedge in a difference between "CC" (your own way of constructing it so as to deny it) and the ecological catastrophe of which carbon emissions are a major driving element is just another form of denial and compartmentalization. The eco catastrophe killing the insects and global warming are the same thing. Or, rather, the global warming brought about by uncontrolled carbon emissions, ocean poisoning and the destruction of primary forest land is a big but hardly the only element constituting the ecological catastrophe. The eco catastrophe is driven by the capitalist organization of political economy. It proceeds from a mode of production that requires prodigious quantities of energy derived from hydrocarbon burning and generates enormous wastes in every stage of the production, consumption and disposal processes. This capitalist organization of economy, which resists planning and public spending if they are unprofitable but loves planning and public spending if they foster GDP growth, requires hydrocarbon burning and generates enormous wastes because these practices generate revenues and profits on the ledgers of enterprises competing for buyers on markets, and thus creates outlets in which the financial sector can invest at an ROI. That is true in China, the United States, Europe, Russia, Japan, the oil-producing countries, India, Latin America, Africa and Oceania. That is real-existing capitalism.

.
We meet at the borders of our being, we dream something of each others reality. - Harvey of R.I.

To Justice my maker from on high did incline:
I am by virtue of its might divine,
The highest Wisdom and the first Love.

TopSecret WallSt. Iraq & more
User avatar
JackRiddler
 
Posts: 16007
Joined: Wed Jan 02, 2008 2:59 pm
Location: New York City
Blog: View Blog (0)

Re: Failure to Archive RI, Feb 2019: My Surrender

Postby Karmamatterz » Mon Feb 18, 2019 11:26 am

Jack,

The best and most fruitful process will first require getting access to the MySQL database that stores all the data. Then determine which fields you want to copy, I would suggest all of them. Basically copy (download) an entire copy of the DB and then import it into a new database. Fields can be mapped exactly as they are or you can remap, which requires a lot more work but there are scripts out there to do this.

What about setting up a simple Go Fund and ask members to donate, then hire a developer who can do this work? I would be willing to help find a developer, there are a lot of them who do freelance work.

Who has full admin access to RI and all its code?
User avatar
Karmamatterz
 
Posts: 828
Joined: Sun Aug 19, 2012 10:58 pm
Blog: View Blog (0)

Re: Failure to Archive RI, Feb 2019: My Surrender

Postby JackRiddler » Mon Feb 18, 2019 12:50 pm

.

Uh, that is so not a thing I can do. I mean, I know there's a thing called MySQL and it's really really important but. Uh... I was sort of hoping for a standard site-sucker program that downloads every page and converts it to static html, changing all the internal links so that the pages still connect to each other. Like happened with the last archive, apparently perfectly and in just 1.2GB downloadable for unzipping and voila, there's the whole thing functioning on your own hard drive.

Such a simple thing and when I describe it I feel like a moron and like I was probably taken for a ride by the buggy softward of Microsys. I will try another site-sucker, just not right away.

Gotta take that break--i already violated my resolution a final time, I hope, but with some research that may be worthwhile on an outfit called "More in Common."

Anyway, thanks!
We meet at the borders of our being, we dream something of each others reality. - Harvey of R.I.

To Justice my maker from on high did incline:
I am by virtue of its might divine,
The highest Wisdom and the first Love.

TopSecret WallSt. Iraq & more
User avatar
JackRiddler
 
Posts: 16007
Joined: Wed Jan 02, 2008 2:59 pm
Location: New York City
Blog: View Blog (0)


Re: Failure to Archive RI, Feb 2019: My Surrender

Postby DrEvil » Mon Feb 18, 2019 5:39 pm

So, I've made a couple of tries with HTTrack, but gave up when it reached almost seven gigabytes in size. I'm pretty sure I managed to filter out all the pictures so I'm not sure why it reaches that size when the 2016 archive was just over three gigs. What I did manage to download seems to work just fine though. With my luck I was missing two posts before completion when I hit cancel.
"I only read American. I want my fantasy pure." - Dave
User avatar
DrEvil
 
Posts: 4143
Joined: Mon Mar 22, 2010 1:37 pm
Blog: View Blog (0)

Re: Failure to Archive RI, Feb 2019: My Surrender

Postby MacCruiskeen » Mon Feb 18, 2019 6:21 pm

In 2019 one would think (one being me, a near-complete programming-illiterate) that there must be a simple and reliable way to download an entire website. Thousands of people must have tried to do so by now, most of them presumably in vain. Yet there is very little doubt that the NSA can do it, for, instance, and does so routinely. Everything online is easily captured, by some, whatever its size and complexity.

Why isn't there something like "convert2pdf" for whole websites? That it takes time is not the issue. That there are apparently no dependable instructions for how to do it -- that is the puzzling thing (puzzling at least to me). Why is there no supply to meet the demand? Why isn't some young genius making his first million by creating EasyWebsiteDownload™?

Can it possibly be that the Free Market™ does not have our best interests at heart? :shock: That is a scary thought.

Anyway, enough with such wacked-out tinfoil-hatted conspiracy-nuttery™. Thanks again to JackR for his efforts, and to Joao for his . (Jack, apologies if I'm asking a question you've already answered, but have you asked Joao how he actually went about it?)
"Ich kann gar nicht so viel fressen, wie ich kotzen möchte." - Max Liebermann,, Berlin, 1933

"Science is the belief in the ignorance of experts." - Richard Feynman, NYC, 1966

TESTDEMIC ➝ "CASE"DEMIC
User avatar
MacCruiskeen
 
Posts: 10558
Joined: Thu Nov 16, 2006 6:47 pm
Blog: View Blog (0)

Re: Failure to Archive RI, Feb 2019: My Surrender

Postby JackRiddler » Mon Feb 18, 2019 7:50 pm

.

Hi MacC.

Off topic, but I'm not posting here any more except related to this problem, and meanwhile, oh my god, this:
https://www.nytimes.com/2019/02/18/worl ... trump.html

Not good.

Now, you people are just not going to let me leave easy, here, eh? (Temporary but firm decision, need to focus elsewhere, too addictive to produce instant writing here and get instant feedback.)

Yes, I tried the same program, and when it failed, I tried the same older version of it Joao used. When I stopped the last time, it was at EIGHTEEN GB (seven days!) and still didn't have large parts of the site, BUT apparently had many pages 10 times over (thus explaining the size). It's possible some update to phpBB in the last two years has made a board with 40,000 threads and however many 10s of thousands of pages be more like a maze to the scanning element, so that it keeps going in circles over the same files but misses others. (I don't know if that sounds like I know what I'm talking about, but I don't. Just guessing. Or making shit up.)

We are talking hundreds of thousands of internal links and hundreds of thousands of external to confuse the poor thing. Remember, it has to CHANGE all of the internal links so that they work once downloaded to your machine, where the address is going to be a folder on your hard drive, not http://www.rigorousintuition.ca/etc/etc.

Also, phpBB is in php, as the name says. That means pages are built off a database every time you load one. (Is that the MySQL? Also, if anyone actually knows what the fuck I think I'm talking about, did I get that right?) For an archive, it needs conversion to html, which is a static format. (In straight HTML the page is the actual saved file, rather than being built off a saved database as in php, see? I should say, I think.)

So it's not that simple and a minor mistake or incompatibility with the phpBB format in a freeware package that doesn't have the staff support to test it and fix bugs constantly can give rise to this problem of bloating on repeat files and missing others. (It's not like Windows, which has enormous staff support to test it and make new bugs because there are too many cooks and too much legacy bullshit code, and justify their existence and the regular roll-out of new versions to force on everyone by adding superfluous features and useless time-consuming steps to whatever it was you were able to do easily before, and new moronic warnings to pester you with whenever you want to pick your nose -- ARE YOU SURE?! -- and new ways for the whole fucking thing to crash and require reformatting twice a year, etc. By the way, stuff like this makes me more OPTIMISTIC about the future, i.e. that we are not going to be replaced by immortal doppelgangers quite as soon as Ray Kurzweil hopes.)

No, THEYtm aren't responsible for the archive problems. I mean, I did download loads and loads of site. It's backed up on the host, presumably.

Drew is the guy who knows the stuff and I believe still fixes tech problems when they arise. Also, Willow knows stuff. (Sounds funny: Willow, she knows THINGS.)

.
We meet at the borders of our being, we dream something of each others reality. - Harvey of R.I.

To Justice my maker from on high did incline:
I am by virtue of its might divine,
The highest Wisdom and the first Love.

TopSecret WallSt. Iraq & more
User avatar
JackRiddler
 
Posts: 16007
Joined: Wed Jan 02, 2008 2:59 pm
Location: New York City
Blog: View Blog (0)

Previous

Return to General Discussion

Who is online

Users browsing this forum: No registered users and 157 guests