Create a migration script from Events to Discourse Events

Open Source
,
#1

Hi guys, finally, I’ve come up with a very simple migration. Its NOT ready for production, but a starting point which I’ll continue to build upon and with you guys’ feedback, we can create a perfect migration.

Currently Supported Fields

event_start
event_end
event_all_day
event_rsvp
event_going

Usage:

  • Use the branch migration. If your setup is hosted, use this line in your app.yml. - git clone -b migration https://github.com/paviliondev/discourse-events.git.

  • Create the new discourse events while keeping the old events data intact : rake events:migrate_events

  • Create the new discourse events and get rid of the old events data: rake "events:migrate_events[1]"

Features:

  • Migrates event_start and event_end
  • All day events start from beginning of the day and last till the end of it.
  • Migrates RSVP users as Going.

Quirks

  • The allowedGroups setting will be defaulted to trust_level_0 for now.

Field map

event_start Unix time stamp: of the event start date/time
event_end, Unix time stamp: of the end start date/time
event_all_day, boolean : whether its an all day event or not
event_timezone, Event’s timezone(Rails style)
event_rsvp, Whether users are allowed to rsvp
event_going, RSVP user ids
event_going_max, max number of people allowed to rsvp
event_version, iCal event version

Unable to migrate

  • category_custom_fields
    • events_enabled
    • events_agenda_enabled
    • events_calendar_enabled
    • events_required
3 Likes
#2

If it’s OK, I’ll join the discussion @fzngagan.

1 Like
#3

Sure @Bletch

1 Like
#4

@fzngagan To kick us off could you start a list of the fields we need to map?

@Bletch Thanks for joining in! Perhaps you could have a think about how the migration script itself would work. I’m not sure a db migration would be the best, as you’d have to make too many assumptions about the user’s environment.

1 Like
#5

event_start Unix time stamp: of the event start date/time
event_end, Unix time stamp: of the end start date/time
event_all_day, boolean : whether its an all day event or not
event_timezone, Event’s timezone(Rails style)
event_rsvp, Whether users are allowed to rsvp
event_going, RSVP user ids
event_going_max, max number of people allowed to rsvp
event_version, iCal event version

these are the values events plugin stores

#6

There are also a bunch of category custom fields that control the way events are handled and configured at the category level.

I don’t know if these have equivalents in Discourse Event.

category_custom_fields: {
  events_enabled: false,
  events_agenda_enabled: false,
  events_calendar_enabled: false,
  events_required: false,
},

Sorry Angus, I’m not sure what you mean by a migration other than a DB migration.

Are you making a distinction between a Discourse plugin/rake task that understands the context of Discourse config/settings, etc. and a SQL-only migration? If it’s the former then that’s my starting point.

A couple of caveats to start with.

Firstly, my use is widespread but in a very specific way. For example, I do not use the RSVP functionality so I feel you’ll need to rope some other users into the project for that side of the plugin. Secondly, I haven’t even looked at Discourse Event yet so I don’t have a view of the delta or the possible workarounds that might be needed.

Let me install Discourse Event on my beta site and I’ll take a look so I can get a feel for what’s required - for my needs.

Dan.

1 Like
#7

@all I’ve started a spec in the OP as a wiki. Please add further details there as you determine them. Anyone is allowed to edit it. The right sidebar is this layouts widget. If you find it too busy you can hide it by clicking the minus -.

@fzngagan We’ll need to map those fields against the relevant fields in the Discourse Events plugin.

This will probably need to be a rake task, as it’ll need to be run when the site admin is ready.

@how Feel free to add anything in here. If you could provide a relevant dataset that would be great too.

1 Like
#8

I’m starting to take a look on how the official event plugin works overall. Will post my findings here.

As for the import script, I’m picturing it to be shipped with our events plugin and a similar UI to discourse backup screen which displays the current status of the process too.

1 Like
#9

@Bletch
A few things that came into mind after using the official plugin

  1. Category settings - No changes required
  2. All the events will be open
  3. RSVP users will be migrated
  4. Event title will be defaulted to topic title
  5. All the rsvp users will be Going

Simplest way to accomplish this migration would be to add event markdown to the beginning of the first post. This will add the event data without us needing to go deep into the mechanism of how events are created by the plugin.

#10

Hi there,

I stumbled upon an error at signup. The account is validated, and the user redirected to the home page and bam. It seems to be related to discourse-events but I suspect it’s a remainder, as an artifact, since neither the new integrated discourse-events nor the original one are activated.

The error says NoMethodError (undefined method join’ for {}:Hash)`

It seems that some artifact remains in the database for loading the plugin that breaks in this particular instance (for new users) but not afterwards. At first I thought it was related to the wizard since I had a wizard upon signup, but no, after removing the wizard the bug remains.

Oh, here’s one for you: although discourse-events is not activated in the settings, in the plugin list it appears as enabled. discourse-calendar is not activated and not enabled.

Please tell me if you need more info.

Here’s the backtrace:

activerecord (6.0.3.3) lib/active_record/relation/finder_methods.rb:342:in `raise_record_not_found_exception!'
activerecord (6.0.3.3) lib/active_record/relation/finder_methods.rb:451:in `find_one'
activerecord (6.0.3.3) lib/active_record/relation/finder_methods.rb:433:in `find_with_ids'
activerecord (6.0.3.3) lib/active_record/relation/finder_methods.rb:69:in `find'
activerecord (6.0.3.3) lib/active_record/querying.rb:21:in `find'
activerecord (6.0.3.3) lib/active_record/core.rb:167:in `find'
plugins/discourse-events/plugin.rb:155:in `block (2 levels) in activate!'
lib/plugin/instance.rb:259:in `public_send'
lib/plugin/instance.rb:259:in `block (2 levels) in add_to_class'
plugins/discourse-events/plugin.rb:171:in `block (2 levels) in activate!'
(eval):50:in `_fast_attributes'
active_model_serializers (0.8.4) lib/active_model/serializer.rb:468:in `rescue in attributes'
active_model_serializers (0.8.4) lib/active_model/serializer.rb:455:in `attributes'
active_model_serializers (0.8.4) lib/active_model/serializer.rb:480:in `_serializable_hash'
active_model_serializers (0.8.4) lib/active_model/serializer.rb:359:in `serializable_hash'
active_model_serializers (0.8.4) lib/active_model/serializer/associations.rb:124:in `block in serialize'
active_model_serializers (0.8.4) lib/active_model/serializer/associations.rb:123:in `map'
active_model_serializers (0.8.4) lib/active_model/serializer/associations.rb:123:in `serialize'
lib/freedom_patches/ams_include_without_root.rb:57:in `include!'
active_model_serializers (0.8.4) lib/active_model/serializer.rb:368:in `block in include_associations!'
active_model_serializers (0.8.4) lib/active_model/serializer.rb:367:in `each_key'
active_model_serializers (0.8.4) lib/active_model/serializer.rb:367:in `include_associations!'
active_model_serializers (0.8.4) lib/active_model/serializer.rb:362:in `serializable_hash'
active_model_serializers (0.8.4) lib/active_model/serializer.rb:343:in `as_json'
app/controllers/application_controller.rb:457:in `serialize_data'
app/controllers/application_controller.rb:466:in `render_serialized'
lib/topic_list_responder.rb:16:in `block (2 levels) in respond_with_list'
actionpack (6.0.3.3) lib/action_controller/metal/mime_responds.rb:214:in `respond_to'
lib/topic_list_responder.rb:9:in `respond_with_list'
app/controllers/list_controller.rb:279:in `block (2 levels) in <class:ListController>'
actionpack (6.0.3.3) lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
actionpack (6.0.3.3) lib/abstract_controller/base.rb:195:in `process_action'
actionpack (6.0.3.3) lib/action_controller/metal/rendering.rb:30:in `process_action'
actionpack (6.0.3.3) lib/abstract_controller/callbacks.rb:42:in `block in process_action'
activesupport (6.0.3.3) lib/active_support/callbacks.rb:112:in `block in run_callbacks'
app/controllers/application_controller.rb:357:in `block in with_resolved_locale'
i18n (1.8.5) lib/i18n.rb:313:in `with_locale'
app/controllers/application_controller.rb:357:in `with_resolved_locale'
activesupport (6.0.3.3) lib/active_support/callbacks.rb:121:in `block in run_callbacks'
activesupport (6.0.3.3) lib/active_support/callbacks.rb:139:in `run_callbacks'
actionpack (6.0.3.3) lib/abstract_controller/callbacks.rb:41:in `process_action'
actionpack (6.0.3.3) lib/action_controller/metal/rescue.rb:22:in `process_action'
actionpack (6.0.3.3) lib/action_controller/metal/instrumentation.rb:33:in `block in process_action'
activesupport (6.0.3.3) lib/active_support/notifications.rb:180:in `block in instrument'
activesupport (6.0.3.3) lib/active_support/notifications/instrumenter.rb:24:in `instrument'
activesupport (6.0.3.3) lib/active_support/notifications.rb:180:in `instrument'
actionpack (6.0.3.3) lib/action_controller/metal/instrumentation.rb:32:in `process_action'
actionpack (6.0.3.3) lib/action_controller/metal/params_wrapper.rb:245:in `process_action'
activerecord (6.0.3.3) lib/active_record/railties/controller_runtime.rb:27:in `process_action'
actionpack (6.0.3.3) lib/abstract_controller/base.rb:136:in `process'
actionview (6.0.3.3) lib/action_view/rendering.rb:39:in `process'
rack-mini-profiler (2.2.0) lib/mini_profiler/profiling_methods.rb:85:in `block in profile_method'


actionpack (6.0.3.3) lib/action_controller/metal.rb:190:in `dispatch'
actionpack (6.0.3.3) lib/action_controller/metal.rb:254:in `dispatch'
actionpack (6.0.3.3) lib/action_dispatch/routing/route_set.rb:50:in `dispatch'
actionpack (6.0.3.3) lib/action_dispatch/routing/route_set.rb:33:in `serve'
actionpack (6.0.3.3) lib/action_dispatch/journey/router.rb:49:in `block in serve'
actionpack (6.0.3.3) lib/action_dispatch/journey/router.rb:32:in `each'
actionpack (6.0.3.3) lib/action_dispatch/journey/router.rb:32:in `serve'
actionpack (6.0.3.3) lib/action_dispatch/routing/route_set.rb:834:in `call'
lib/middleware/omniauth_bypass_middleware.rb:68:in `call'
rack (2.2.3) lib/rack/tempfile_reaper.rb:15:in `call'
rack (2.2.3) lib/rack/conditional_get.rb:27:in `call'
rack (2.2.3) lib/rack/head.rb:12:in `call'
lib/content_security_policy/middleware.rb:12:in `call'
lib/middleware/anonymous_cache.rb:354:in `call'
rack (2.2.3) lib/rack/session/abstract/id.rb:266:in `context'
rack (2.2.3) lib/rack/session/abstract/id.rb:260:in `call'
actionpack (6.0.3.3) lib/action_dispatch/middleware/cookies.rb:648:in `call'
actionpack (6.0.3.3) lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
activesupport (6.0.3.3) lib/active_support/callbacks.rb:101:in `run_callbacks'
actionpack (6.0.3.3) lib/action_dispatch/middleware/callbacks.rb:26:in `call'
actionpack (6.0.3.3) lib/action_dispatch/middleware/actionable_exceptions.rb:17:in `call'
actionpack (6.0.3.3) lib/action_dispatch/middleware/debug_exceptions.rb:32:in `call'
actionpack (6.0.3.3) lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
logster (2.9.4) lib/logster/middleware/reporter.rb:43:in `call'
railties (6.0.3.3) lib/rails/rack/logger.rb:37:in `call_app'
railties (6.0.3.3) lib/rails/rack/logger.rb:28:in `call'
config/initializers/100-quiet_logger.rb:23:in `call'
config/initializers/100-silence_logger.rb:31:in `call'
actionpack (6.0.3.3) lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
actionpack (6.0.3.3) lib/action_dispatch/middleware/request_id.rb:27:in `call'
lib/middleware/enforce_hostname.rb:22:in `call'
rack (2.2.3) lib/rack/method_override.rb:24:in `call'
rack (2.2.3) lib/rack/sendfile.rb:110:in `call'
actionpack (6.0.3.3) lib/action_dispatch/middleware/host_authorization.rb:76:in `call'
plugins/discourse-prometheus/lib/middleware/metrics.rb:17:in `call'
rack-mini-profiler (2.2.0) lib/mini_profiler/profiler.rb:246:in `call'


message_bus (3.3.4) lib/message_bus/rack/middleware.rb:61:in `call'
lib/middleware/request_tracker.rb:176:in `call'
rails_multisite (2.5.0) lib/rails_multisite/middleware.rb:25:in `call'
railties (6.0.3.3) lib/rails/engine.rb:527:in `call'
railties (6.0.3.3) lib/rails/railtie.rb:190:in `public_send'
railties (6.0.3.3) lib/rails/railtie.rb:190:in `method_missing'
rack (2.2.3) lib/rack/urlmap.rb:74:in `block in call'
rack (2.2.3) lib/rack/urlmap.rb:58:in `each'
rack (2.2.3) lib/rack/urlmap.rb:58:in `call'
unicorn (5.7.0) lib/unicorn/http_server.rb:632:in `process_client'
unicorn (5.7.0) lib/unicorn/http_server.rb:728:in `worker_loop'
unicorn (5.7.0) lib/unicorn/http_server.rb:548:in `spawn_missing_workers'
unicorn (5.7.0) lib/unicorn/http_server.rb:562:in `maintain_worker_count'
unicorn (5.7.0) lib/unicorn/http_server.rb:295:in `join'
unicorn (5.7.0) bin/unicorn:128:in `<top (required)>'
vendor/bundle/ruby/2.6.0/bin/unicorn:23:in `load'
vendor/bundle/ruby/2.6.0/bin/unicorn:23:in `<main>'
1 Like
#11
#12

I just stumbled upon this. How did you deal with this then. Did you have to get rid of the plugin?

#13

@Bletch
I’m refocusing myself on this one for the coming weeks. Happy to get this migration in.

1 Like
#14

Hi guys, finally, I’ve come up with a very simple migration. Its NOT ready for production, but a starting point which I’ll continue to build upon and with you guys’ feedback, we can create a perfect migration.

Currently Supported Fields

event_start
event_end
event_all_day
event_rsvp
event_going

Usage:

  • Use the branch migration. If your setup is hosted, use this line in your app.yml. - git clone -b migration https://github.com/paviliondev/discourse-events.git.

  • Create the new discourse events while keeping the old events data intact : rake events:migrate_events

  • Create the new discourse events and get rid of the old events data: rake "events:migrate_events[1]"

Features:

  • Migrates event_start and event_end
  • All day events start from beginning of the day and last till the end of it.
  • Migrates RSVP users as Going.

Quirks

  • The allowedGroups setting will be defaulted to trust_level_0 for now.
1 Like
#15

Great! What’s a useful way to help with testing this?

If I enable this plug in and also Discourse Events on a test instance, should I just run rake “events:migrate_events[1]” and then check for problems?

1 Like
#16

@TallTrees
The code is sitting on the migration branch currently. Thanks for the nudge. I’ve updated the instructions accordingly.

2 Likes
#18

This is starting to look good now. I’ve implemented the logic to migrate rsvp data.

2 Likes
#19

Sorry @fzngagan for letting you down on this one. Honestly it’s way too far in the past for me to remember how I dealt with it. I think I had disabled the plugin (my last bug report should confirm that it’s still disabled). Worth noting that I stumbled upon a similar issue with a disabled plugin interfering with Discourse: Multi-site upgrade leaves _some_ sites unavailable - #5 by hellekin - support - Discourse Meta. I think this should be dealt with globally in the Discourse plugin API, forbidding a disabled plugin to run code at all.

1 Like
#20

No worries.

Me and all the discourse devs wish this would be the case but its a bit tricky to achieve this.