Streamlining Google Data Takeout for Google+
Just posted at #PlexodusReddit, based on the discovery (and some further experimentation) that it's possible to specify Google products on the Data Takeout request URL itself.
Generating and using Google Data Takeout is complicated. We've simplified one part of the process by specifying the products you need to take out to create a complete Google+ archive on the URL. And that would be:
https://takeout.google.com/settings/takeout/custom/circles,plus_communities,plus_one,plus_pages,profile,stream
As always, when following third-party links to Google account services, verify that the link is what it appears and does what it says.
We're providing the bare, rather than a formatted link, to make this clearer.
What this link does is go to the standard https://takeout.google.com site, and then specifies a custom request, with the following sources ("products") selected:
* circles: Your Google+ Circles. Essentially contact information.
* plus_communities: Your Google+ Communities. This is metadata only, not a full export of your community posts and comments.
* plus_one: Your "+1"s on external websites. (A long-since discontinued service.)
* plus_pages: Any G+ Pages associated with your account.
* profile: Your G+ Profile page and information.
* stream: Your G+ posts, comments, polls, and other content.
Note that this is intended as a comprehensive G+ archive specification. You may not have content in all these categories and you may not be interested in all of this content. But as a starting point for whittling down, it is simpler and far less confusing than the numerous additional Google data archive products listings in the default page.
Not included are:
* plus_photos: Google+ photos, exported by album. These are already included by posts, though not organised by albums.
We've included the Data Takeout link in the /r/plexodus sidebar.
For Google's official support documentation, see: https://support.google.com/plus/answer/1045788
Update: 14 December - removed plus_photos from URL.
https://old.reddit.com/r/plexodus/comments/a5qyyl/streamlining_google_data_takeout_for_google/
So: I'm trying to improve on the Google documentation. I'm not sure I have. I've written too much, I always do.
ReplyDeleteCritical editorial eyes to simplifying the fuck out of this would be hugely appreciated.
Thanks.
Please note. plus_photos: Google+ photos, exported by album.
ReplyDeleteThis is actually Google Photos. Google+Photos is hidden inside G+Stream
And yes, moving G+Photos to G Photos is a FAQ.
Julian Bond I don't believe I ever had Google Photos, though I posted a bunch of photos to G+. Did those then become Google Photos?
ReplyDeleteAnd should plus_photos be included or removed from the URL spec? Will any G+ content not be included if that's pulled?
First. Remove Plus_photos from the URL. It adds an entry that is not strictly to do with G+ Removing it doesn't remove any G+ Specific entries.
ReplyDeletehttps://photos.google.com/ is where all your android phone, and Drive photos end up. It has history in G+photos and Picasa. A long time ago, there used to be a mechanism to move G+Photos to G Photos but that's long since gone.
G+Photos are photos specifically uploaded for inclusion in G+ Posts. When you do that you can pick from your existing G-Photos or upload a new one. That uploaded new one doesn't get put into G-Photos. It's just attached to that post. To get hold of these in Takeout. DON'T Takeout G Photos. Instead go to G+Stream, select data source and check "Photos".
Yet another example of where Google's history of products and partial rationalisations results in a bloody mess.
Julian Bond Interesting. Seems that my last uploaded photos were from April 7, 2016, which was probably when Photos split from G+. A quick perusal says that these are all images I'd uploaded to G+, not separate Photos uploads, and not other incidental images from my Android devices -- which I've kept on other, nonsensical, identities, and think I've disabled auto-photo-uploading from.
ReplyDeleteJulian Bond Is there a selection for Activity? "My Activity" perhaps?
ReplyDeleteThe other product that's been suggested be added is Contacts, which might help resolve G+ Circles or other user information, though I'm not sure of that.
Julian Bond Note also: "not strictly" and "assures completeness" are not the same thing. The goal here is to have a simple URL that ensures a complete G+ archive. That may mean it includes more items than is necessary, but ensures it doesn't have fewer than are needed.
ReplyDeleteI'm going to try to get more clarity on this point. I do appreciate your comments. Abundance of caution, etc., etc.
Check Takeout.G+Streams. In there is a checkbox "Select Specific Data". Choose that and then click on the button "Google+ Stream data". You then get a popup with Activitylog, collections, events, photos, posts. Having done the download, Activitylog contains +1s on comments, +1s on posts, comments and poll votes.
ReplyDeletehttps://lh3.googleusercontent.com/BBYVWhXo1VE1RAJ1bFf7AQi14tBsI8NN1pvZnfj2TtVbc6tEenqBOZ5g-Hcomt0hZR0T4T9-SvUzX43nP60HvB6i2J-Pm0mBDMw=s0
Check Takeout.G+Streams. In there is a checkbox "Select Specific Data". Choose that and then click on the button "Google+ Stream data". You then get a popup with Activitylog, collections, events, photos, posts. Having done the download, Activitylog contains +1s on comments, +1s on posts, comments and poll votes.
ReplyDeletehttps://lh3.googleusercontent.com/2Hz4iBlXqqTQ4DCqJUygniKpIItjBD4MJIW_QO23SdKvEbsnL67RK7Y64Js1vXrXWA7NW3Kg2Sf0IYDCu5d70Lm2ZqX3GDsgwuI=s0
Check Takeout.G+Streams. In there is a checkbox "Select Specific Data". Choose that and then click on the button "Google+ Stream data". You then get a popup with Activitylog, collections, events, photos, posts. Having done the download, Activitylog contains +1s on comments, +1s on posts, comments and poll votes.
ReplyDeletehttps://lh3.googleusercontent.com/gB4SKXKKIoPw6xWGpcBhPixuWaSZpgtQmYBUftZ1x1XLsYWs7TwaeT5OACkQnHVj2Y4PIjY3h-qfUOh0YoUrWEDC8spvcu304Tg=s0
Check Takeout.G+Streams. In there is a checkbox "Select Specific Data". Choose that and then click on the button "Google+ Stream data". You then get a popup with Activitylog, collections, events, photos, posts. Having done the download, Activitylog contains +1s on comments, +1s on posts, comments and poll votes.
ReplyDeletehttps://lh3.googleusercontent.com/yKJhgwKbB8N797zUpK9JQe6fk8upsSlo6Ji74UP3RbTgk_6CEeFdE2HzMrFC7Fctd93IauSDJAhQmiQ-_GPkDo2GaCWCNp76sfM=s0
Check Takeout.G+Streams. In there is a checkbox "Select Specific Data". Choose that and then click on the button "Google+ Stream data". You then get a popup with Activitylog, collections, events, photos, posts. Having done the download, Activitylog contains +1s on comments, +1s on posts, comments and poll votes.
ReplyDeletehttps://lh3.googleusercontent.com/gEAh9ol2YRd7NYRyCJTxApElMjjiEqcZxNyGn7mTpmSu56wRtp57NVWxnN09qXEHATVhreqp6v_9XeyH-_aRX_HwKr7WeMy-QIs=s0
Check Takeout.G+Streams. In there is a checkbox "Select Specific Data". Choose that and then click on the button "Google+ Stream data". You then get a popup with Activitylog, collections, events, photos, posts. Having done the download, Activitylog contains +1s on comments, +1s on posts, comments and poll votes.
ReplyDeletehttps://lh3.googleusercontent.com/rQBPkiOdYGHdF-q4kJOjH5V6Iq-obp5jE3T6Eqd7X0iTRWoGt02-FeUydDZ3Z7xQCZKtCVjsqHrRnd28-3BwHj-cUgCkxBbyHAs=s0
Check Takeout.G+Streams. In there is a checkbox "Select Specific Data". Choose that and then click on the button "Google+ Stream data". You then get a popup with Activitylog, collections, events, photos, posts. Having done the download, Activitylog contains +1s on comments, +1s on posts, comments and poll votes.
ReplyDeletehttps://lh3.googleusercontent.com/kwIodBle8Iv45gMmRytAQT6qRjL-taOEjfE5lPayhvoBkAxzoLiqEZwjuynZDlDAAyeaJs-2kEEAoty7vXXmbJ5DFmNotcEIxGg=s0
Check Takeout.G+Streams. In there is a checkbox "Select Specific Data". Choose that and then click on the button "Google+ Stream data". You then get a popup with Activitylog, collections, events, photos, posts. Having done the download, Activitylog contains +1s on comments, +1s on posts, comments and poll votes.
ReplyDeletehttps://lh3.googleusercontent.com/aEEkNBTM6qMO967rYilSD_iJB4oY6-kt_O1Da3Zd2N9ZfzwkvdTtUUCb9JHd7Gs6gG_HKDo2tbh4Eof2GR1ISO7DVf47CQdFFVQ=s0
Check Takeout.G+Streams. In there is a checkbox "Select Specific Data". Choose that and then click on the button "Google+ Stream data". You then get a popup with Activitylog, collections, events, photos, posts. Having done the download, Activitylog contains +1s on comments, +1s on posts, comments and poll votes.
ReplyDeletehttps://lh3.googleusercontent.com/c_1Irt0zAL0SnsBF2aFjkzNGFQaZ6Ctj6WN8sa1tarKiy4TiwneS4lkGj3wTF8OoCT2AuVcXzqp3w5lLdufJW3hthR6Jp84xwpI=s0
Check Takeout.G+Streams. In there is a checkbox "Select Specific Data". Choose that and then click on the button "Google+ Stream data". You then get a popup with Activitylog, collections, events, photos, posts. Having done the download, Activitylog contains +1s on comments, +1s on posts, comments and poll votes.
ReplyDeletehttps://lh3.googleusercontent.com/0FI4kz123OilxWiHTBGS2Z2jaj5G_7lTR_4nweWLkg1KRzvS6AGIpe8mysUh3yj7MZxbyTju5436Pyobg1UZ9lMl7H0Wx6t9rAA=s0
Check Takeout.G+Streams. In there is a checkbox "Select Specific Data". Choose that and then click on the button "Google+ Stream data". You then get a popup with Activitylog, collections, events, photos, posts. Having done the download, Activitylog contains +1s on comments, +1s on posts, comments and poll votes.
ReplyDeletehttps://lh3.googleusercontent.com/5LbRono_dzHkE3mzTJ7LWlyf3foED_obpuHXEUfi4McxUEXmoASsEwobnDuwCo04H6dJspMUhqYaYVisu2MNZd-aMCHrisTC7OE=s0
Check Takeout.G+Streams. In there is a checkbox "Select Specific Data". Choose that and then click on the button "Google+ Stream data". You then get a popup with Activitylog, collections, events, photos, posts. Having done the download, Activitylog contains +1s on comments, +1s on posts, comments and poll votes.
ReplyDeletehttps://lh3.googleusercontent.com/EYWoOl9IqjuwCaKUsih35iqiLB4hcsQKo9r80kQwklE599ayKP7OSWWeUQ902AKHKIT-fKjLlEzdQrIQjX9w-s5Jlfu92roF8Ew=s0
Check Takeout.G+Streams. In there is a checkbox "Select Specific Data". Choose that and then click on the button "Google+ Stream data". You then get a popup with Activitylog, collections, events, photos, posts. Having done the download, Activitylog contains +1s on comments, +1s on posts, comments and poll votes.
ReplyDeletehttps://lh3.googleusercontent.com/YlwRhn-4OUSM_Xl2p5Ts2X4REU-NLd3wKEkDGTMcbggvaaKdlrvYNiDZ5_tm1WUdFc-QYoMWFm9ZgX9am-YxcvBRhh0ZhJcH-FY=s0
Check Takeout.G+Streams. In there is a checkbox "Select Specific Data". Choose that and then click on the button "Google+ Stream data". You then get a popup with Activitylog, collections, events, photos, posts. Having done the download, Activitylog contains +1s on comments, +1s on posts, comments and poll votes.
ReplyDeletehttps://lh3.googleusercontent.com/jWdgKB4LhXk6KVWBMoBVug_oLzyGPLg3zYNZ2XIdoG1l3lYukoMPmdjq-j6nr6TxgenzfeMtgwXCOqP4wiw5GZc7ZX7G3c_Xelc=s0
Check Takeout.G+Streams. In there is a checkbox "Select Specific Data". Choose that and then click on the button "Google+ Stream data". You then get a popup with Activitylog, collections, events, photos, posts. Having done the download, Activitylog contains +1s on comments, +1s on posts, comments and poll votes.
ReplyDeletehttps://lh3.googleusercontent.com/8y_rAt44pTOJbhK5NtX4g0jK83VkySriXw6IqkyTcpc41exV53bJqpfbRA4Z_4d-sBH-yQ8Lop41mo5jJsKKUM9Hk8SKyO2ucsg=s0
Check Takeout.G+Streams. In there is a checkbox "Select Specific Data". Choose that and then click on the button "Google+ Stream data". You then get a popup with Activitylog, collections, events, photos, posts. Having done the download, Activitylog contains +1s on comments, +1s on posts, comments and poll votes.
ReplyDeletehttps://lh3.googleusercontent.com/NK-rgh3ThbIf3g5eUELl-iXkIEqswVleHloVTuOM3DwYW5I9a04qwXOa2VmEqfo9doctW0IW7S-5yRAsUrtBDAKsz9k-rZ0yZOg=s0
Check Takeout.G+Streams. In there is a checkbox "Select Specific Data". Choose that and then click on the button "Google+ Stream data". You then get a popup with Activitylog, collections, events, photos, posts. Having done the download, Activitylog contains +1s on comments, +1s on posts, comments and poll votes.
ReplyDeletehttps://lh3.googleusercontent.com/fJJjwO1CaBy97z6US2ZyMBRcP1Ow0DxkTYnulLXtC2jSgmuU5dvIrfUMSljz6-TNABhVcIrRY3RKuPjF3NhGtxWwkDZuCMp12Xw=s0
Check Takeout.G+Streams. In there is a checkbox "Select Specific Data". Choose that and then click on the button "Google+ Stream data". You then get a popup with Activitylog, collections, events, photos, posts. Having done the download, Activitylog contains +1s on comments, +1s on posts, comments and poll votes.
ReplyDeletehttps://lh3.googleusercontent.com/_KlJzXBgJxvIytoeoRX-4AAeJRS4hdsEVNRq1rmtczkAoYx_PhVHkcRk0NLupITiFrpQjshbS257Jn6QLVXiaD5sLmCAhKBcJgU=s0
Check Takeout.G+Streams. In there is a checkbox "Select Specific Data". Choose that and then click on the button "Google+ Stream data". You then get a popup with Activitylog, collections, events, photos, posts. Having done the download, Activitylog contains +1s on comments, +1s on posts, comments and poll votes.
ReplyDeletehttps://lh3.googleusercontent.com/xtGNsuFkwCEjsRm_1h8FAn_bZ62BSnvDnCg-klFL4fjnISDqDsB6fO0MICeFmKTz8t_ncnBh-Js803y7w8CJpT_4QIY3wKMw_0I=s0
Check Takeout.G+Streams. In there is a checkbox "Select Specific Data". Choose that and then click on the button "Google+ Stream data". You then get a popup with Activitylog, collections, events, photos, posts. Having done the download, Activitylog contains +1s on comments, +1s on posts, comments and poll votes.
ReplyDeletehttps://lh3.googleusercontent.com/ObbIxpgfpsCp1Y-4NaUCUK1OUqBSYwaSf7oCvpfK3RfQw7ztoNJ5WdeSv4ZOj5uI8Qn_Y4ZGO0mfW7I9GyZ4DXw9Gwnj0QwR1Gk=s0
I haven't yet done a Takeout.Contacts. Trying it now. I would expect it to be all GMail/Android Contacts. it might include circles or not. The problem is I've already exported G+circles via Takeout and then imported those files into Contacts, so it may be hard for me to tell.
ReplyDelete[Edited to add]
The only entry in the Takeout.Contacts big contacts VCF file for you is identical to the one marked as Imported on 27-Oct-2019. And it's here.
BEGIN:VCARD
VERSION:3.0
FN:Edward Morbius
N:Morbius;Edward;;;
EMAIL;TYPE=INTERNET;TYPE=WORK:spacecat@gmail.com
NOTE:Display Name\: Edward Morbius\nProfile URL\: https\://plus.google.com/
104092656004159577193\n
PHOTO:https://lh5.googleusercontent.com/-X5KWOZXJ-_Y/W_-OLdTEurI/AAAAAAAAAA
A/JodGIPdRwMQF4uJoSZQymBrENK0TQYr6QCOQCEAE/photo.jpg
END:VCARD
So my take on that is that Takeout.Contacts doesn't automatically include G+Circles contact data. So you do need to export G+Circles and then import those into Contacts. However the data is minimal. Several things. I see that your public work email address is in there. I'm not at all sure where that came from. Also that your avatar photo is in there. along with a link to your Google+ Profile. Will those still exist or 404 after the G+Sunset?
So I'm still not sure what the hell is going on here!
Julian Bond Checking just now: "Streams" selects all data by default, including Activity Log. So that's set.
ReplyDeleteThough you've got to go through and convert each to JSON rather than HTML format. Sigh.
Regarding Google Contacts, I don't think Google+ Circles have ever exactly been part of "Your Contacts" - they definitely aren't now.
ReplyDeleteYou can move people in your Circles in Contacts to "Your Contacts":
1. Sign in to contacts.google.com on desktop
2. If using Contacts Preview, click Switch to old version on the left menu
3. Click a Circle
4. Select all the people in the Circle
5. Click the Groups icon at the top of the page
6. Click the box next to "My Contacts"
I'm guessing that after you do that those people will now be in the Contacts archive.
Julian Bond Looking at that contact card, I notice it does not include any forwarding information or existing Web contacts -- if people are relying on being able to scrape forwarding contact information from exported G+ Circles data, they will be disappointed.
ReplyDeleteAnd I've got to update my email address....
Peggy K I do know that for a time my Gmail contacts were getting seriously polluted with G+ contacts. I think that may have been reversed. Made Gmail all but useless for a while though. Likely in the 2013 - 2015 timeframe.
ReplyDeleteEdward Morbius And I've got to update my email address ;) Nope. I modified it to avoid posting what I wasn't sure was public.
ReplyDeleteJulian Bond Gmail isn't my primary. I'd seen that elsewhere earlier as well. On my Very Long List....
ReplyDeleteI have tried to export my G+ Stream using Google Takeout. I have tried *four times* now and each time, it ends with "Archive incomplete" after 1 day of exporting. What should I do? Each of the "incomplete archives" has completely different size.
ReplyDeleteFrantišek Fuka this doesn't fix it for you, but do send Feedback to Takeout by clicking the ? at top right on your Manage Archives page, and select "Send Feedback". And you may also want to post in the official Google+ Help Community, as specific reports there can be compiled to pass on to Google+. The more details you provide the better.
ReplyDeleteFrantišek Fuka What Peggy K said. Feedback's not an ideal channel, but it gets received. We're compiling and forwarding information as well.
ReplyDeleteGoogle Takeout for G+ data is a joke. I've left Feedback about all the problems I have. All I want to do is download the posts from 3 of my collections. Can't do that. If I select to download my Stream, it gives me 32GB of data. I've selected only Posts (no images) but it's still 32GB. I know I have not posted 32GB of text.
ReplyDeleteMy slow rural DSL Internet connection would take several days to download this much data even if Google's servers pushed it at my max download rate, but Google is trickling it out at about 0.1 MB/s which would take weeks (other sites can stream to me at full bandwidth so this is a Google problem, not an ISP problem).
EDIT: After several hours the download speed picked up to my max download speed, so the first of my 17 2GB files should be finished in another hour. Sigh.
Their infrastructure is broken and they don't give me the tools to download a reasonable subset of my data.
I considered trying that third party tool that's been mentioned here. But I'm leery of running an unsigned executable by an unknown author from an anonymous Dropbox folder that Windows then says is untrusted.
?
ReplyDeleteFrantišek Fuka I would download it regardless of it being marked as 'incomplete'.
ReplyDeleteOnce you've downloaded it, you can open the Takeout/index.html in the archive to get an overview of the files it didn't manage to include.
They might also link directly to the online versions of those files, so you can try to download them manually.
If the missing files are all in a single subset of Stream, for instance ActivityLog, you can also try requesting just that subset again.
If the same files keep popping up as incomplete, you can report those specific errors via the Send Feedback form from the top right triple vertical dot menu in the desktop version of Google Takeout.
I examined the first 2GB file of G+ Stream data I downloaded. OMG what a disaster. No wonder it's so frigging huge. First, it's mostly photos (I specifically deselected the photos checkbox because I just want the post text). Most of them are duplicated, up to 30 times. I don't need 30 copies of the same image. Many of the images are not mine. Apparently it downloads all posts and images that I have been tagged in and/or commented on. Also multiple duplicates of metadata files. I don't need 30 copies of the same metadata.
ReplyDeleteWhat a frigging useless waste of bandwidth. I just want the posts I created, not ones created by other people, without photos (I already have those on my hard drive, thank you) and only one copy, not 30.
I already left this feedback to Google. Here, I'm venting. I will not download any more as it's a useless waste of time, bandwidth and disk space. I hope Google plans to make this more useful.
Alan Bland your feedback here on issues is welcome. :) So, feel free to rant. :)
ReplyDeleteFilip H.F. Slagter Thanks! I unpacked that zip, deleted the duplicate files, and that 2GB file only had 500MB of unique content. A substantial fraction of that content were jpg files that I did not recognize, certainly not images i ever posted.
ReplyDeleteAlan Bland What FiXato said. Very valid feedback.
ReplyDelete