Development plan & progress log

Alpha

Delivers a simple D7 site based on TN's own new distribution and module set, with component/plugabble (initiative) widget system, internationalisation capabilities.

Research tasks

• TODO update IIRS/SolutionDesign
• UNDERWAY: Investigate I18n & related language/internationalisation modules & best practice -- see ​https://drupal.org/node/133977 and ​https://drupal.org/project/tmgmt
• POSTPONED: Resolve tension/shortcoming in nested Drush Makefiles causing module dupes between /profiles/modules and /sites/all/modules - ​https://drupal.org/node/1006620
• DISCOVER: Investigate content syncing tasks based on Services and Services Client modules, check also Deployment modules

Plan

1. UNDERWAY Finish view page:
   • Map view/search -- views based again, using Leaflet or Leaflet Map Box plus ​IP Geolocation Views and Maps.
2. Override view output to pop up result in another DIV/iFrame. -- is mostly done via CTools/Panels with minimal coding. -- various options: ​https://developer.mozilla.org/en-US/docs/Web/API/Window.postMessage, ​https://drupal.org/project/ajax_pane, ​http://drupal.stackexchange.com/questions/50453/drupal-using-panels-with-ajax and ​http://www.gizra.com/content/views-and-panels-ajaxified/
3. Export TN-specific stuff to Features. Update all features.
4. Set up fieldable pane for About page/tab and export content.
5. Set up test site on TN's server (Puffin) -- also allows us to set up/test Hybrid Auth social media login.
6. Update views, panes, etc to handle language context.
7. Build some admin views to aid moderation and translation.
8. Code refactor & Drupal standards cleanup.
9. Clean up & refactor modules, commit to Git.

Other key tasks

1. Add Services and Services Client modules
2. Debug & improve the base Hub install profile.

Nice to haves

These are a secondary goals that are nice to haves, but the IIRS widget with internationalisation and associated (syncing) directory is the primary goal... To do if we have time, Ed to prioritise as needed.

1. Move 'iFrame' specific code to own Widget Framework plugin with key theme and utility functions.
2. Enhance iFrame code with responsiveness? ​http://www.advomatic.com/blogs/dave-hansen-lange/responsive-iframes-yes-it-possible
3. Create 'Hub Profile' on clone (STG?) of D6 TN.org contains basic text, country, language and widget displays.
4. [8] (Ben?) Update widget HTML/CSS/theme as needed.
5. Add Optional CSS/style options on Widget Builder page.
6. The ability to import/export a built widget via CTools & Features

Beta

Delivers a cleaned/tweaked up system and some web services connections back to the D6 site, plus an installed and integrated beta website.

1. [12] Fix bugs/do tweaks
2. [32] Add Services to sync between D6 <--> D7 sites (and future hub sites) with resilient queuing, plugable architecture
3. [12] Test, tweak, prep Beta, install, integrate

Subtotal: 6.5 +/- 3 days (= 52 - 76 hrs)

Total: 18.75 +/- 8 days (= 150 - 216 hrs)

NB: from Ed - Jim - your total time budget is 150 hrs. This gives us 3 days with a second programmer in beta phase (agreed as good idea). Bear this in mind!

Development Log

Notes

• Hours in [square brackets] is an estimate, (hours in normal brackets) on done items is time actually spent.
• 21.5 hours already spent on design, meetings and documentation

26 August -- 4 hours spent

• Clone winacc_panpoly distribution repo -> rename to transitionnetwork_d7 - ​transitionnetwork_d7 repo (1.5 hours)
• Change profile to be TN-focused, include key modules: (1 hours)
  • Address, Geofield, geocoder, geophp
  • Leaflet
  • dnl, i18n
• Set up local development site (1.5 hours).

28 August -- 3h (7h total)

• R&D into
  • various aspects: BOA, Leaflet/OpenLayers?, makefiles etc (0.5 hours)
  • R&D into internationalisation dnl, i18n (0.5 hours)
• Create Initiative content type with required fields (0.5 hours)
• Create Base and Initiative feature, push to GitHub? (1.5 hours) -- ​Base repo, ​Initiative repo

29 August -- 4.5h (11.5h total)

• Social login decided, added to makefile: HybridAuth? Social Login (0.5 hours)
• R&D for user profile modules, added user profile2 module (0.5 hours)
• Ed Mitchell & Jim Kirkpatrick Skype PM/Design/Progress meeting (1 hour)
• Looked into: ​Countries, ​Location Taxonomize and decided to KISS for now. (0.25 hours)
• R&D/Decision Made: NO user location-related fields present on basic Drupal account, First, Last & Middle name fields only. ALL location and other features for users will be on Profile2 and defined later. (0.25 hours)
• Added realname and realname_registration to remove need for username on registration (0.25 hours)
• Create User Profile feature (decide approach Profile2 vs core user fields etc) (0.75 hours) -- ​User repo
• Updated base and user features to include required modules, began Social Login work with Hybrid auth -- will wait until test server setup as oAuth keys etc needed (0.5 hours)
• Investigate I18n & related language/internationalisation modules & best practice -- see ​https://drupal.org/node/133977 (0.25 so far)
• Resolve tension/shortcoming in nested Drush Makefiles causing module dupes between /profiles/modules and /sites/all/modules - ​https://drupal.org/node/1006620 (0.25 so far)

31 August -- 0.75h (12.25h total)

• Module & widget design sketching, R&D... Decisions made: (0.25 hours)
• Debug & improve TN D7 Hub install profile. (0.5 hours)

3 September -- 4h (16.25h total)

• Module & widget design sketching, R&D... Decisions made: (3.75 hours)
  • Try to use as much of Panels and Page Manager as possible for rendering, building and managing widgets.
  • New structure follows;
    • Framework = Defines the wrapper, renderer, libraries (CSS, JS) and means of operation of the widget. Frameworks must be able to render the key set of elements that make up a widget. For this project we're only interested in a 'Simple HTML' framework. Future frameworks could include: AJAX HTML, iGoogle, Backbone.js, Angular.js etc.
    • Layout = A way of arranging widget components (Panels panes) on the widget. For this project we'll build 'Full page', 'Sidebar' and 'Button only' widget layouts.
    • Elements = a chunk of widget that will be delivered by the framework -- the bits a widget is built from with the framework... We need: Container (shows a panels display in iFrame), Modal-container (shows panels display iFrame in popup), Button, Link... More can be added as needed and provided by other modules etc.
    • Workflow = A means of linking displays (pages) and processes within the widget together.
  • ​http://drupal.org/project/widgets is no use at all -- good module for in Drupal site widgets, not for generating/rendering those outside.
• Email regarding integration with incumbent/existing sites (0.25 hours)

4 September -- 1.5h (17.75)

• Development Work to create plugins and framework using panels/ctools as base (1 hour)
• Skype with Ed regarding progress and plans (0.5 hours)

Hiatus, retool

Ed moved house, Jim was busy... Lots of thinking happened but not much actual development. Jim is now back and all over this like a {insert suitable simile‎ here}!

22 October -- 1 h (1 total)

• Mild rethink to maximise results, minimise development and aim squarely at the IIRS widget as the MVP. -- DOCS UPDATE PENDING --
• Plan of module & architecture from Drupal/code perspective -- code sketching.
• Decision taken to not include panoply

24 October -- 4h (5 total)

• Ed & Jim project Skype meeting
• Created and enhanced Widgetizer module based on previous code.
• Added new CTools type for 'Combined login/register' pane, (re)using much of PSE code for that.
• Fixed annoying typo/bug preventing Ctools seeing new types.

25 October -- 1h (6 total)

• Added stub Ctools types for entry page for entity... At this point unsure if I'll use the basic/standard Initiative Node form, or a separate form with mapping (per PSE)

3 November -- 5h (11 total)

• R&D: decided CTools multistep form wizard isn't up to the job -- too focused on forms, can't handle multiple steps. Now need to see if a module or interface exists for handling multi pages with workflow type progressions -- Options remaining:
  1. (original plan) Simple CTools plugin type that allows code to specify steps and criteria
  2. A panels region that uses position/index of pane to decide position in workflow, or same principle with Page and it variants -- hacky, but might be quick and dirty...
  3. Another module, TBC.
• Built proximity search based on Geofield and Views -- can search for any postcode/address/city+country the user enters. Ideally add HTML5 geolocation, too -- ​https://drupal.org/node/2011340
• Now need to make map views --liking the look of ​https://drupal.org/project/ip_geoloc for searchable map views a LOT.
• Also interesting: ​https://drupal.org/project/global_filter -- allows a user to set their location ONCE and it be applied to all views needing location henceforth. Hmm..
• Lots of research into possible modules for workflow... Either too specific, not D7, or just not a good fit -- SO, decision made, will write a simple and editable CTools Workflow plugin that can handle our needs.
• Wrote the guts of the Workflow plugin, now is able to handle the pages and route the request accordingly.. Need to add configuration and/or subtypes to allow for each content type to be handled cleanly.
• Primary PoC added to Ini CT.

6 November -- 3h (14 total)

• Got workflow working, can now wrap up the steps required to get a user from login/register -> Submit X content -> Thanks page.
• Started building 'Widget' entity type -- checking whether to use ECK module or just do in code. This will allow the context of the widget (type, owner, language, workflow etc) to be held and new types made.
• Skype with Jim & Ed.

7 November -- 1.5h (15.5 total)

• Hit snags around current planned approach, needs simplifying. Issue is basically that own Widget Instance entity plus separate workflow/framework/container PLUS all the UI stuff... is going to be a LOT of work. So one last change of design is needed to get to an MVP ASAP, whilst being a flexible as possible. Consider issues over weekend and alter tack slightly.
• Question around if Widget instance is even needed? This is the part that might be good if we want to allow lots of users to have widgets AND track them, but for the IIRS needs it's overkill. A better approach will be to put the required variables inside the URL, and use those to pass to the entry workflow and views.

11 November -- 1.5h (15.5 total)

• Played around with Panels Tabs and Panels Ajax Tabs. Neither worked as I'd have liked, but Panels Ajax Tabs has most promise as it lazy-loads the panes as the user changes between them. Quicktabs was used for the PSE but it forces much of the config and choice into code. Hit a bug or two in Panels Ajax Tabs but will try further.
• New, simplified plan is:
  1. Build an 'Entry' page in panels. The url will be [language]/tn-widgets/entry/user-submit/[step], and the [step] is an argument that will be passed to the Workflow I've already created. [langauge] will be looked after by Drupal.
  2. Will create a 'Workflow Container' content type that is a pane that has the following settings form:
     • Workflow - a select list to pick the appropriate workflow. To start there will be one: user login/reg, submit content, thanks page.
     • Step - takes [step] argument from the URL (see above) and passes it to the Workflow module so it knows what step page to show
     • Content type - the choice of content to submit for the 2nd step
     • Thank you message - A WYSIWYG field that is the thankyou message. Ideally needs to be translatable in the UI.
  3. Will create an entry panels page with URL [language]/tn-widgets/view/country/[country-code], meaning the [country-code] will be used to do the filtering. Language is separate from country (though connected in real life) and present by the [language] argument.
  4. The widget container/framework/layout that puts these above two pages into other web pages will remain a CTools plugin, but will need its own UI for getting the widget. I'm looking at modules to do this for me, otherwise I'll re-use the PSE code as much as possible.

12 November -- 1h (16.5 total)

• Quickly converted the PSE to D7 to aid in grabbing/reusing code -- especially around the 'Get Widget' page.
• Reviewed many modules and CTools best practice..

13 November -- 5.5h (22 total)

• Identified the ​Multilingual Panels module which solves a big issue... Allows settings for panes (e.g. thank you text) to be translated from within Panels.
• So DECISION MADE Establish whether to use existing, standard Panels renderer or customise new one for working with Widgets. Will use Multilingual Panels as this has its own renderer. All my rendering will be done by Panels, except the actual output will go into an iFrame provided by the Widget Layout/Framework?.#
• Also, DECISION MADE will use use current page manager thus avoiding having to write any UI, just some forms to store config.
• Refactored code to match the above plan.
• Updated plan and log.
• Build Workflow Container and have *almost* got it working exactly as needed. Just need to get the title passed through, and the configuration of the underlying Workflow saved/loaded properly. The admin UI is all groovy though.
• Installed the French and Portuguese languages, downloaded their .po files from ​https://localize.drupal.org and tested various pages in these languages. All works well now ready to begin preparing the list of strings for translation.

14 November -- 4.5h (26.5 total)

• Workflow Container works, and now the conf is stored, but the saved settings aren't being added to the form when editing again.
• Workflow access fixed.
• Workflow can now can deliver combined user/login page, then entry page showing correct node form (initiative). The last page is easy.
• Ajax address field stuff is presently broken. Am working on submission process now as I'm several levels deep.
• Multilingual test works -- forms all translate (where strings exist) correctly, node saved in correct language.
• Have found ​Translation Management Tool module that looks to make translation of site content really easy.
• Started update of IIRS/SolutionDesign, which has reminded me that we need a country argument in the URL for the entry widget page... This will make it look like: [language]/tn- widgets/entry/initiative/[country]/[step]. According to ​this Countries module issue this is now possible via said module. ​Countries cropped up several times in my travels (e.g. 29 Aug), so will add to the mix right now.
• Hit issue around country argument.

15 November -- 4h (30.5 total)

• Fixed addressfield ajax issue, will put a patch on D.o.
• Also had to use dev version of Countries, but now the URL for the entry is: tn-widgets/user-submit-national/gb/initiative/entry - the 'initiative' part allows the same Panels page have many variants for the content types we want to support, whilst the 'entry' is the step we're on and 'gb' is the country code to use. Also the URL starts with 'fr/' when in French and 'pt-pt/' for Portuguese... This is all we need!
• Working on final issues and bugs around workflow container/pane admin screen so we can store the thanks text.
• Entry nearly working end-to-end. Multilinguality great, happy Jim.
• Except the redirect after form submit is being overridden by other things - so the workflow's submissions aren't sending to the next step.

16 November -- 1h (31.5 total)

• Debugging, refactor to better use Drupal's drupal_static() function and avoid passing around everything. Tidy up.

17 November -- 4h (35.5 total)

• Continued refactor, improvements, bug fixes.
• Managed to fix the redirect issue after stealing some code from ​Login Destination since I was battling with many core issues/features.
• Workflow process and container now work (login -> submit -> thank you page), all except for the $conf from the workflow setup page not being saved properly.
• 'Thanks' screen showing with hard coded text, will fix after the map and popup view panels are done.

18 November -- 1.5h (37 total)

• Work on final bug whereby the pane config is not completely saved.
• Skype with Ed.

19 November -- 2.5h (39.5 total)

• Fixed conf bug, got entire Workflow process to work and messages to show. Entry widget pane/panel is now 90% done.
• Improvements pending refactor

20 November -- .5h (40 total)

• Enabled Multilingual Panels and proved config is translatable for our Workflow pane. Works well!

21 November -- 1.5h (41.5 total)

• ​Development estimates and build overview for TN.org v3
• Skype with Ed
• Begin refactor and commit to Git

24 November -- 3h (44.5 total)

• Got Widget Style plugins working and builder built.
• Many small issues left, need to build a lightweight 'iframe' Framework plugin to render common parts of the needed Widgets Style.

25 November -- 1.5h (46 total)

• Code tidy, debugging, work to make Builder page better.

26 November -- 1.5h (47.5 total)

• Skype with Micheal and Hannes regarding the German needs, IIRS and future TN.org engagement with the world-wide techies.
• Skype with Ed to discuss IIRS project.
• INVOICE SENT: 47.5 hours resetting counter on log.

27 November -- .5h (.5 total)

• Updated log.

29 November -- 1h (1.5 total)

• Started 'iFrame' framework plugin and related functions - mainly theme and helper functions that allow the new Widget Styles that are based on iFrames to be implemented cleanly.
• Started settings form for each 'widget_style' plugin to allow to be used in the Build page.

1 December -- 5h (6.5 total)

• Finish 'Build Widget' page!
• Settings for widget_style included with defaults too. Theme() functions used and working, all dynamic and plugable.
• NEEDS REFACTOR!!
• Imported & tweaked CSS & JS from PSE to handle widget popup/style.
• Widget can be previewed and code grabbed on builder page - tested and works well.

2 December -- 4h (10.5 total)

• Got Panels AJAX Tab working on the display pane, but have hit ​a few bugs in that module. It nearly works very well for us.
• Started a ​tn_widgets module that will hold the key TN-specific widget settings, form alters and logic to switch theme within the widget URLs.
• Preparing to update the makefile so we can deploy a test environment on the TN server (Puffin).
• Skype with Ed & Jim

3 December -- 1.5h (12 total)

• Refactor, improvements and bug fixing on Builder.

4 December -- h (+12 total)

• Creating map view... Some issues found and working through...