The Egg timer application has a number of features:
-
Reference data - e.g. Dissolution periods, Parliamentary periods, Sessions, Houses
-
A calendar view showing whether either house is sitting, or adjourned, scrutiny non-sitting days etc.. along with whether each house is praying or not.
-
Calculuate sitting days during an interval
-
Documenting the different available periods
-
A guide to events, what to do a prorogation and dissolution etc
-
Checks around whether the synchronisation processes are working or not
The egg timer synchronises data from a few different sources:
-
The parliamentary periods and sessions spreadsheet - this has two tabs
- Parliament periods
- Sessions
-
The following public google calendars:
-
Lord's virtual days calendar - the Lords continue to have virtual days, the Commons do not.
These calendars are currently owned by a couple of users using their own google account.
A postgres database is used to store the calendar events for the website.
The application simply reads the various environment variables set up with the google credentials in Heroku. For running locally, a .env file is required with the credentials.
The calendars are synchronised using the simple ruby Google API client (RESTful). The 'modern' client does not provide a calendar endpoint. The simple endpoints are still supported by Google.
Authorisation is by using a standard google service account - this is the recommended way for using this ruby API.
The latest synchronisation date is stored, so that only entries after that date are added to the database.
These are imported from the CSV files in db/data using the setup.rake task
The calendar uses the sync token functionality provided by the API so that the client can request any changes since the last sync, rather than having to do a full refresh.
The app stores the sync token once one of the calendars has synchronised.
Once all of the calendars have synchronised, a CalendarSync record is created which then provides a timestamp of when the last succesful run was complete.
Sometimes a synchronisation will fail with this message from the API:
Google::Apis::ClientError: fullSyncRequired: Sync token is no longer valid, a full sync is required. (Google::Apis::ClientError)
So, mapping the cycles of this:
For that calendar, the api will be called and any updates will be pulled down from the Calendar and the database updated accordingly.
The google API call throws something like a Google::Apis::ClientError exception during the get_changed_events_from_calendar method. When this happens we create message, mark the sync as unsuccessful, created the DetailedSyncLog entry and send an email that there has been an issue
When a synchronisation has failed, the application will attempt to auto-fix it.
Syncing::GoogleCalendar contains the generic synchronisation code and we store the details in the DetailedSyncLog (and corresponding table).
When generic_sync is called for a calendar, with it's corresponding handler, we first check to see if there are any unsuccessful syncs. If there is one, we get the broken sync log (ready to update later).
Then using the handler, we delete everything for that house, so all SittingDay for example, we find the SyncToken for the calendar id and delete it.
We then update the broken sync log to say we were succcesful.
As the calendar has no SyncToken, a full download will be triggered
There is a guide to resetting the database in Prorogration and Dissolution or online
Ideally we will use the test environment credentials for development, but we can pull the database from Heroku (see below)
Then we need a .env file in the application route to set up our environment variables - these will be picked up automatically when the rails app is started, as we use the dotenv gem to locally read the .env file.
If running against test, then copy the credentials set on the test environment for the Google auth fields
GOOGLE_ACCOUNT_TYPE=service_account
GOOGLE_CLIENT_EMAIL=
GOOGLE_CLIENT_ID=
GOOGLE_PRIVATE_KEY=
# Production
# COMMONS_SITTING_DAYS_CALENDAR='20n14bks46tvd2k5rse3jmsfb4@group.calendar.google.com'
# LORDS_SITTING_DAYS_CALENDAR='o26tfi8b5o78cborja7utgpcb8@group.calendar.google.com'
# LORDS_VIRTUAL_SITTING_DAYS_CALENDAR='p1lfs3elv1fk0lqdigs3jngop8@group.calendar.google.com'
# COMMONS_ADJOURNMENT_DAYS_CALENDAR='ikdqq0rcg07bbs64g7aeqqlkt4@group.calendar.google.com'
# LORDS_ADJOURNMENT_DAYS_CALENDAR='ibbc1cen1mdm6rsf6kkno17i0c@group.calendar.google.com'
# COMMONS_RECESS_DAYS_CALENDAR='eefeb6980f4ee93bd3d486b318141524452c82b8388066ef868e3443a549e3c3@group.calendar.google.com'
# LORDS_RECESS_DAYS_CALENDAR='45591a2f31eb089019ba1b200e5ec635f8d25a9620f120e96e881b3165e714d4@group.calendar.google.com'
# Test
COMMONS_SITTING_DAYS_CALENDAR="08c47a45c4ad926a2ed5690d5de6664ec4ed5f25ee77e3cbab1bd2b3b54cd804@group.calendar.google.com"
LORDS_SITTING_DAYS_CALENDAR="321431db1e84f9c1743f5b1c3291a1544fbda486f8ccc2d1a021b448598de92e@group.calendar.google.com"
LORDS_VIRTUAL_SITTING_DAYS_CALENDAR="902c32b23639d887c56c9a2cef82c9d0dc352d3e5967bc4806062dbf917920d5@group.calendar.google.com"
COMMONS_ADJOURNMENT_DAYS_CALENDAR="9dc2afe4c49604de9a88e836449a49a90356e2796f626d5aa896fc6f64c6fab8@group.calendar.google.com"
LORDS_ADJOURNMENT_DAYS_CALENDAR="d3333dda8d0fc131f470bc145c5d6b9fedc9a449827fc3dcf485290ce123819b@group.calendar.google.com"
COMMONS_RECESS_DAYS_CALENDAR="c168720cd26912a9f1f872e5c87d41809b5ffd0ec08194c23bf9ceca17b8f043@group.calendar.google.com"
LORDS_RECESS_DAYS_CALENDAR="843571d922480c5de55c8a354e31eb59061a58b11ff9fca2b8bd154a62bf7b92@group.calendar.google.com"
For production
heroku pg:pull DATABASE_URL parliament_calendar --app parliament-egg-timer
For test
heroku pg:pull DATABASE_URL parliament_calendar --app egg-timer-test
If you want to drop a local db first:
dropdb parliament_calendar; heroku pg:pull DATABASE_URL parliament_calendar --app egg-timer-test
Using commentariat (does this still work?):
ruby commentariat.rb -s lib/ -o demo
... where "demo" is the output directory.