Migrating communities from Google+ to Discourse
I wrote an import script to move Google+ content into Discourse, and used it to import about a dozen Google+ communities into the forum.makerforums.info Discourse instance. Feel free to look at that site to see what the import looks like, and to join if you are a maker!
Importing Friends+Me into Discourse preserves both Google login and community content with continued ownership of that content in the new community for users who choose google login to their Discourse forum as at least one authentication method.
For more information, see the original request for this functionality at https://meta.discourse.org/t/bounty-google-private-communities-export-screenscraper-importer/108029 and ask questions there so that they survive the gpluscalypse.
[Updated 7 March to reflect current status.]
https://meta.discourse.org/t/bounty-google-private-communities-export-screenscraper-importer/108029/11
I wrote an import script to move Google+ content into Discourse, and used it to import about a dozen Google+ communities into the forum.makerforums.info Discourse instance. Feel free to look at that site to see what the import looks like, and to join if you are a maker!
Importing Friends+Me into Discourse preserves both Google login and community content with continued ownership of that content in the new community for users who choose google login to their Discourse forum as at least one authentication method.
For more information, see the original request for this functionality at https://meta.discourse.org/t/bounty-google-private-communities-export-screenscraper-importer/108029 and ask questions there so that they survive the gpluscalypse.
[Updated 7 March to reflect current status.]
https://meta.discourse.org/t/bounty-google-private-communities-export-screenscraper-importer/108029/11
Hmm, since I do have experience with Ruby, that bounty might be interesting.
ReplyDeleteI'll let you figure that out with respect to the other interested developer, and my offer to share work in progress at whatever time someone else wants to use it as a basis for development stands for anyone seeking the bounty, if I don't get so far that the bounty closes first. The bounty is clearly too low to really cover costs for someone doing this professionally; I'm working on it for the joy of rescuing data from the gpluscalypse.
ReplyDeleteI think I have most of the guts of user import written including merging with existing google accounts and setting up for users to be able to authenticate and merge accounts later to take ownership of posts, and I'm working on importing posts and comments. My writeup there has a lot of the useful data.
The bulk of my known remaining work before starting what will probably be an epic debugging project is primarily translating from my python script linked from my comment in that Discourse post to Ruby. I'm converting titles differently because Discourse topic titles should be shorter than what I came up with in python, but the formatMessageElement function has the core needs for message formatting handled.
I also haven't yet added a function to add tags based on categories in the import data, which is desired for the first import I want to run.
/sub
ReplyDeleteDiscourse makes the message formatting relatively easy.
ReplyDeleteI'm still pulling together post/comment (in discourse, that's topic/post) import and haven't started image import.
I also need to change it to be able to import lots of imports together, importing all the G+ users across all the community export files first, and then import all the communities. That way, references to users across communities can work right.
Filip H.F. Slagter I should clarify: I'm offering to share my work in progress with anyone who asks, regardless of whether they are seeking the bounty. With respect to timing, I'm quite sure I won't have it finished this weekend, and next weekend I am fully otherwise committed, so realistically I'm looking two weeks out before I have a reasonable hope of working code.
ReplyDeleteMichael K Johnson I'm currently occupied with family arrangements too, so I unfortunately am lacking time too...
ReplyDeleteMy importer is working. I'll be doing a PR soon.
ReplyDeleteMichael K Johnson nice job with creating such a tool. I didn't use it yet, but it looks promising. Could you please tell us what's the scenario of creating topics in Discourse during importing process? I mean does [g+community].post equals [discourse].topic and [g+community].post.comment eq [discourse].topic.post? Or perhaps one can customize this relation before running the tool?
ReplyDeleteNot only can you customize it, you must the way I have written it. If you haven't already explicitly mapped communities to categories and subcategories, it writes a JSON mapping file, raises an error, and stops. The comments at the top of the script show how to do that. That mapping also allows you to assign tags on a per-category basis, and applies the "gplus" tag to every imported topic.
ReplyDeleteThe mapping doesn't have to be 1:1 and onto. You can map G+ categories to tags if you want. Some of the G+ communities I imported to forum.makerforums.info went into a discourse subcategory of a larger discourse category, and I used tags in discourse to identify the G+ category from which the topic was sourced.
Terminology is hard here, because a G+ post is a discourse topic and a G+ comment is a discourse post, and internally in discourse both topics and posts are posts. If that sounds confusing, imagine writing the importer and keeping it in mind! ;)
If you have more questions, please ask at meta.discourse.org - [bounty] Google+ (private ) communities: export screenscraper + importer - marketplace - Discourse Meta so that the answers survive the gpluscalypse.
Updated the top post with current status to make it easier for the next person who has a question... ☺
ReplyDelete