I hope this saves a few of you some time and aggravation.
Originally shared by Edgar Brown
I carried out my last data export from G+ and with little more than 24hrs to go, here is my experience. In hopes that it will help some of the procrastinators..
1450 posts more than 6000 comments, first post in February 2012. I was obviously into cameras back then.
Forget about Google TakeOut in any format. But particularly in html format.
It will save the text of your posts and some information around it, but most of it will be gone with G+, as it contains links to G+ addresses. This is also true for the Json format.
I ended up using Google + Exporter (and shelling out the $20 because I have more than 800 posts in my stream). But, although the archives are more complete and the export controls are much more detailed than Google's Takeout, that is still not enough. As many of the media links head back into g+.
After trying at the several formats offered, I ended up choosing Wordpress for the main export (but also did Json for good measure, as G+exporter makes that quick and easy). The actual download took ~20min but once it is there all the data is local and ready to be exported.
I ended up creating a local installation of a Wordpress site, as my platform is a mac I can only speak for that. It took about 15 minutes to install the local server software and Wordpress and get a working configuration. Although I mostly know my way around a server I had never used Wordpress or MAMP before (I did not even have to open a console this time).
I had to divide the downloaded data into 500-post chunks, as the final xml export, at 14MB exceeded the Wordpress import limit of 8MB. Thankfully that's just an option in the exporter.
When you import into Wordpress it gives you the option to bring in the linked media with the import. So you have a fully-working local site. It also gives you the option to associate the import to different users, so I could very easily separate my pages and communities.
It is not perfect, but at least now I have a searchable well-formatted archive of the vast majority of my G+ history. (And also of some of the communities that I moderated).
It took me ~6hrs from start to finish (including realizing that I could not do much with the Takeouts and the multiple tries for some of my mistakes and a bit of Wordpress customization (e.g., adding a dynamic user directory page).
Good luck.
See you in the bitstream....
https://gplus-exporter.friendsplus.me
https://www.themeum.com/install-wordpress-localhost/
#gpmigration
F+m export to wordpress has also been the best for me. However I didn't realize you could run a local server for it; I just used the website.
ReplyDeleteThanks for this post. I'm going to try the local server thing.
Last night I tried to Takeout a single private community. All that came down was a page full of URLs to G+. This is not a data takeout. So I used the G+ Exporter and imported it into Blogger and WP.com successfully, though WP didn't offer to bring in linked media. Maybe that only works with externally hosted sites.
ReplyDeleteeric peacock I exported mine into a WAMP server on my PC and it worked great. Have you installed wordpress manually before? If I recall that is what I had to do when I put it on my own server. The WP site has a great "5 minute install" tutorial that will help if you haven't done this before.
ReplyDeleteMichael Russell I'm going to build a Linux machine and install it all tonight. I've never done xamp before, but I have installed WP desktop app (not sure if that's the same thing).
ReplyDeleteSomeone should take the time to create a simple virtual machine package with Wordpress already running on a barebones OS. Mmmm.... I wonder
ReplyDeleteNot that many around...
But this one might work for some. It's site specific but it might be worth the try for some users while you wait for a whole operating system to install...
bitnami.com - WordPress Virtual Machines
trying to get the install completed and found this, linux vm with wordpress installed and configured:
ReplyDeletebitnami.com - WordPress Virtual Machines
eric peacock I don't have much experience with that stuff. The main potential hurdle to the 5 minute wordpress setup is having the databases configured (it isn't difficult) before the install. The 5 minute install I was mentioning is here:
ReplyDeletecodex.wordpress.org - Installing WordPress « WordPress Codex
It does take me 5 minutes now, but was longer when I was still learning how all this worked. Anyway, something to look at if your Wordpress virtual machine doesn't pan out.
I haven't downloaded the VM yet. Following links from the OP, I got started installing xampp Kubuntu and the WP plugin for it. Since I'm not a Linux "poweruser" or whatever, it took a bit to get it going.
ReplyDeleteOnce I got the WP plugin up and running, I started the import from the Friends+Me export.
After I clicked submit, it goes to this page:
http://localhost/wordpress/wp-admin/admin.php?import=wordpress&step=2
It seemed like it wasn't doing anything, so I canceled it and restarted. Not I have a few pages of "post already exists". LoL
Point being, even though it doesn't appear to be doing anything, it is.
I'll update again when it's done.
https://lh3.googleusercontent.com/5jfta6rNCYn337kwGo-WHwRePiwSIE4EaCqe1DBFOJ5Kwi8y3xtPVKfb75BxjpaOq9X1KGZYNXg2Abw=s0
LOTS of these, not promising :( :
ReplyDeleteFailed to import Media “😢😢😢😢😢😢😢😢”
Failed to import Media “Some people don't understand why some mourn the loss of actors, singers, and writers so heavily.”
Failed to import Media “#SQUIRREL”
Failed to import Media “#feelz”
Failed to import Media “#monday”
Failed to import Media “#squirrel”
Failed to import Media “Great pic, these are definitely my favorite Trek Doctors.”
Failed to import Media “Creative👏”
Failed to import Media “Title”
Failed to import Media “Title”
Failed to import Media “Ultimately, it will be up to ordinary citizens and local police officers to stop the civil war that they are trying...”
Failed to import Media “Title”
Failed to import Media “Title”
Failed to import Media “Title”
Failed to import Media “Title”
Failed to import Media “Title”
Failed to import Media “Straight up Truth...”
Failed to import Media “Title”
Failed to import Media “Can anyone find Gary Mitchell?”
Failed to import Media “Yeah, it's all conspiracy theories and paranoia.”
Failed to import Media “Title”
Failed to import Media “Title”
Failed to import Media “Happy blasphemy Sunday everyone ;)”
I had one "failed to import media" in just under 800 posts I imported yesterday on my localhost wordpress install. This may just be a problem between your computer and google, as your computer has to download all the files from Google (or wherever they were hosted) as it adds those images/media to your media library. For me, approximately 500 jpgs that showed in my media library took just over an hour to populate, so do give it some time to get it sorted. Might there be a common theme between the posts with import media failures above?
ReplyDeleteeric peacock I got a lot of those. I did not pay much attention to them.
ReplyDeleteDo remember that this is nearly a decade of posts, a lot of those media files are likely to have been removed by their owners, or the accounts were closed, or what you. For me it was rather common to see an old post with a missing picture every now and again.
Take a look inside G+ at hose posts (you should be able to easily find them inside Wordpress with the title and then use the date and author to find them inside G+) that should put you at ease.
FWIW: VMs nowadays are very straightforward. Their format has been mostly standardized and there are very good open source alternatives for them.
ReplyDeleteThere are a few commercial ones, however I believe the open-source no-frills standard is Virtual Box, and it's as simple as installing an application. Nowhere as complicated as doing a full linux installation.
Then the virtual machine is loaded into the application, and configured via the graphic interface so that it steals just the right amount of your base machine resources.
The downside for something as easy to install like Wordpress is that it might be configured in a particular undocumented way (for example, to get the environment to match the limitations of a specific cloud provider) that might complicate things down the road.
But if anyone has the time to put together a no-frills VM for this specific purpose, it would make life easier for some people.
Does this extract media files from the local archives or does it download them using the G+ links in each post ?
ReplyDeleteYounes Layachi You mean when you import it to wordpress or somewhere else? In wordpress, I looked at a post soon after import and the image had a google url initially. After it all showed up in the wordpress media library, that post had my wordpress url for the image.
ReplyDeleteBe advised. Although Wordpress import detects and avoids duplicate posts, it does not do that with comments. Due to my multiple import passes for importing individual public communities AND my personal stream, some posts ended up with triplicate comments.
ReplyDeleteAlso, my personal stream was tiny compared to those of many people. I started an export from a friend stream and it has been going for more than 5hrs. It's up to 16000 posts and 85000 comments. I doubt that I will be importing media files with that one, that will surely hit my internet quota.
Michael Russell yes when I already have the takeout/export that consists of text/urls + media files, and I want to make it into something readable by using this WordPress method here , does it match every media file to its post or does it have to download every media from scratch using the urls ?
ReplyDeleteYounes Layachi When you import the xml file that the Google + Exporter produces, into wordpress, the importing of the media files is handled for you. The result is a post that looks just like it does on G+, minus some formatting perhaps, and since G+ posts don't have a title, the wordpress titles are taken from the first text in the post. So some cleanup is likely necessary, but the rest is handled automatically.
ReplyDeleteYounes Layachi That's the main problem with all of the export formats, the links are all to internet addresses. AFAIK: Wordpress has no way of knowing what media file goes with what and, given the very short time window that Google provided there are no tools to make this work.
ReplyDeleteWould it have killed Goggle to say:
we are freezing G+ on March 31st 2019 0000GMT, the archived site will be active until July 31st 2019 0000GMT, the data archives will be preserved and accessible until December 31st 2020 0000GMT. In that period of time we will dedicate a couple of underpaid interns to provide you with open-source data migration and visualization tool support. Here is some code and format definitions, have fun.
That would have provided enough time for the development of a reasonable data and folder structure format and tools that would have made our lives much easier.
It is somewhat possible that someone will develop a set of tools that would be able to match the Json html links to the actual files in the hard drive. It is even possible that will find out about it. But I would not hold my breath.
Edgar Brown All I know is that when I imported the xml file from the G+ exporter into Wordpress, the media came with it. So for a G+ post that was some text and a photo, that became a wordpress post with that text, and the photograph. After import that photograph was hosted on the wordpress site, not at a google url, and was placed in the correct post. Maybe that scenario is not what you are referring to?
ReplyDeleteThat said I totally agree that the way Google has handled this whole thing has been subpar, to be kind.
Michael Russell The media must have come directly from the web, not your file system.
ReplyDeleteThere is no obvious way to map internet urls to local file paths, unless there is a very clear agreed-upon schema that I am not aware of.
Edgar Brown In my G+ post the image url looks something like this:
ReplyDeletehttps://lh3.googleusercontent.com/-f1QlSq8mSgg/XJ1ZsMmaTZI/AAAAAAAA2r0/9LxAh21TXWwyJDXnPPjFU9PRpzujfmG0wCJoC/w530-h353-n/pitt-river-addington-marsh-17140.jpg
After using the G+ Exporter tool and importing the xml file into my local wordpress install, the post reappears as a wordpress post, with the same image, but now with the url of:
http://localhost/blog/wp-content/uploads/2019/03/pitt-river-addington-marsh-17140.jpg
Wordpress and the xml file are doing all the work here, nothing manual. I still have the feeling we might not be talking about the same thing. :)
Michael Russell You already have the relevant media files as part of the Takeout export in your file system, right?
ReplyDeleteWordpress, thanks to the links inside the .xml or .json files, has the internet url of the original media files in Google's servers right?
Wordpress knows where to get the information from, it's just not looking into your file system for it. It downloads it (again) and it incorporates it into its own database and local filesystem creating the link that you see.
This can be a problem for those that have very large amounts of media in G+, as they might have to go through the download twice. I predict that more than a few will hit their internet data caps in this process.
Confirmed, double comments on second import attempt
ReplyDeleteConfirmed, WP imports media files from URL in XML file
Comment media is shown as link, media not downloaded.
Have not tried WP import from Takeout data, not sure this would work. Friends+Me export doesn't download media, only produces XML with media links.
eric peacock it should be possible to import to Wordpress from a Google Takeout set of files. You have to install a json import plugin into it (there are a couple of those). But I don’t think there will be a difference.
ReplyDeleteEdgar Brown Doh! Why didn't I think of looking for a json import tool.
ReplyDeleteThanks
Edgar Brown With takeout I guess you would have the relevant media files, but by all accounts unless one has experience parsing through stuff like that - matching them up and recreating the G+ posts from scratch wouldn't be something most people could do. I have my takeout files, but haven't done anything with them. All my import tests have been with the G+ Exporter tool, and that has worked fine and doesn't require much technical knowledge - nor the files to be found locally on your machine.
ReplyDeleteWe're doomed :/
ReplyDeleteSo how does MeWe do it ? They use the links as well ?
mewe.com - MeWe: The best chat & group app with privacy you trust.
ReplyDelete