Transition Technology: Ticket Query

#865: synchronisation

annesley — Wed, 15 Jul 2015 13:26:48 GMT

ideas. please query them.

we are synchronising between different data structures: WordPress and Drupal and anything else the plugin is installed on. therefore standard *database level* distributed synchronisation management tools will not be appropriate. this is unfortunate because synchronisation is a big task. however, it is possible that there are some CRUD / REST based sync tools. so: we need an XML abstraction layer (partially done already) produced by the Drupal, Wordpress, etc. plugin that is standardised and can then be compared and synced via standard API calls.

Steps:

new Transition Town registration on server A notify server B that there is new data and send the GUID of this new data server B then requests only the new data from server A (incremental) using the GUID server B creates the new item in it's database with a new native ID using the abstraction layer in it's plugin / module

addtions to this universal data pool, e.g. a new Transition Town, will be propagated via a network sync request at point of addition. "listener servers" will then request the new data (incremental only) and, in turn push that out to all other listeners. each plugin will therefore extend and expose it's CRUD style synchronisation abstraction functions:

add-user add-local-group change-user etc.

many of these are already available as part of the framework-independent plugin / module

currently, i suggest that ALL plugins contain ALL the international user and Transition Town data. passwords and emails, contact info will be handled by a 3rd server, either Mozilla Persona or Open ID. user accounts will also be synchronised on to ALL plugins but without passwords as those are held on the 3rd server. thus far had already been agreed with Ed. but, ofc, can be changed :)

new plugin installations will receive a full complement of data at time of installation. check digits will be periodically shared to check that all data is in-line. all users will be able to register and edit their data on ANY website holding the plugin. TT and USER changes and registrations will then propagate via PUSH notifications across the entire network all native IDs will be different. i.e. TT Brixton will have a different ID on each server. thus, as always with synchronissation, all IDs will be transformed to GUIDs by the abstraction API and only GUIDs will be used to analyse the network of data and synchronisation. login to any website containing the plugin will be transparent (unlike the demo i set up) through the normal wordpress and drupal login screens. the plugin will intercept failed authentication and attempt to authenticate against the universal servers. new accounts created via universal registration on any server will have a framework specific configurable role and thus permissions on that server will be set by the administrator specific to that server.

#792: [Security-news] SA-CONTRIB-2014-094 - Webform Patched - Cross Site Scripting (XSS)

paul — Mon, 29 Sep 2014 09:28:08 GMT

View online: https://www.drupal.org/node/2344369

Advisory ID: DRUPAL-SA-CONTRIB-2014-094
Project: Webform Patched [1] (third-party module)
Version: 6.x, 7.x
Date: 2014-September-24
Security risk: 13/25 ( Moderately Critical)

AC:Basic/A:User/CI:Some/II:Some/E:Theoretical/TD:Default [2]

Vulnerability: Cross Site Scripting

The Webform Patched module is a fork of the Webform module with Token support added. The module enables you to create forms which can be used for surveys, contact forms or other data collection throughout your site.

The module doesn't sufficiently sanitize field label titles when two fields have the same form_key, which can only be managed by carefully crafting the webform structure via a specific set of circumstances.

This vulnerability is mitigated by the fact that an attacker must have a role with the permission "create webform content".

/A CVE identifier [3] will be requested, and added upon issuance, in

accordance with Drupal Security Team processes./

Webform Patched 6.x-3.x versions prior to 6.x-3.20.
Webform Patched 7.x-3.x versions prior to 7.x-3.20.

Drupal core is not affected. If you do not use the contributed Webform Patched [4] module, there is nothing you need to do.

Install the latest version:

If you use the webform module for Drupal 6.x, upgrade to webform_patched

6.x-3.20 [5]

If you use the webform module for Drupal 7.x-3.x, upgrade to

webform_patched 7.x-3.20 [6]

Also see the Webform Patched [7] project page.

Maurits Lawende [8]
Matt Vance [9]

Nate Haug [10] the module maintainer

Greg Knaddison [11], Dan Smith [12] and Lee Rowlands [13] of the Drupal

Security Team

The Drupal security team can be reached at security at drupal.org or via the contact form at https://www.drupal.org/contact [14].

Learn more about the Drupal Security team and their policies [15], writing secure code for Drupal [16], and securing your site [17].

#741: Views editor disappears in backend

annesley — Thu, 12 Jun 2014 10:42:13 GMT

admin > views > edit the view editor interface appears and then disappears immediately this happens in Chrome / Ubuntu and Firefox / Mac

#857: Tiny MCE weirdness

sam — Tue, 02 Jun 2015 15:24:33 GMT

Hi Paul,

Myself & Rob have both run into an intermittent issue where when editing a panel page the WYSYWG editor (Tiny MCE) sometimes appears, sometimes doesn't.

When it doesn't appear you are left with the plain text html editor.

There seems to be no obvious pattern to it. So might be a tricky one to debug.

I see the version of Tiny MCE we are using is quite old, so I was thinking perhaps we should just try upgrading it on a dev server and see if that fixes it?

If this seems reasonable could you stick the latest Tiny MCE on your dev server so we could test it out there? Or if you have any other ideas for getting to the bottom of it..

Thanks

Sam

#859: Subscription emails broken

sam — Tue, 16 Jun 2015 13:08:44 GMT

Hi just got this mail

"For some reason I realized I wasn't hearing from Rob. You might want to check your system because mine hasn't changed as far as I know."

Had a look in my inbox & the last mail from Drupal subscription system was on 27th of May.

I may be the guilty party, as I did go in to edit the message around this time.

I'll investigate via the Drupal admin interface, but has anything else happened/ been done that could have stopped the mails?

#761: Spam account cull

ed — Thu, 17 Jul 2014 08:45:33 GMT

There are bucketloads of spam accounts swamping us. Spam commeting is swarming again. I just did several pages of deleting spam accounts. No doubt I nailed some humans too (sorry Sam if this comes back to you); but the overwhelming majority of new accounts are spam.

It's crap and we need to have another spam sweep - especially if we're staying in D6 for a while.

See work done in Feb 2013: #461 See wiki page done in Feb 2013: https://wiki.transitionnetwork.org/Spam_accounts

SAM I'm going to suggest you start looking at it, and get your head around it, and the various modules and processes we've got running, then ask you to act/escalate accordingly.

#890: Site offline.

sam — Sat, 12 Dec 2015 10:54:36 GMT

It's serving a page, so may be Drupal level problem rather than server level?

https://www.transitionnetwork.org/

#488: Set up Dev/Test and update CodeManagementReleaseProcess for new Aegir, Git, Drush make approach

jim — Mon, 04 Feb 2013 19:07:31 GMT

This page is now out of date... https://tech.transitionnetwork.org/trac/wiki/CodeManagementReleaseProcess

This ticket is to update this with a new version, and set up Dev and Test environments, documenting all as we go.

#737: SPF / Emails rejected from the website contact form

sam — Thu, 05 Jun 2014 15:46:13 GMT

We had a user report that they could not send a message via our contact form:

"Yesterday I sent a message to you via the contact form on the website. But obviously something went wrong: for I got a failure notice saying my message could not be delivered. Therefore I'm sending it directly via email (see below) hoping that you're receiving my message this way."

<info@…>: host mx1.spamfiltering.com[72.249.150.158] said:

550 81.95.XX.XX is not allowed to send mail from gmx.de. Please see http://www.openspf.net/Why?scope=mfrom;identity=userXX@gmx.de;ip=81.95.XX.XX (in reply to end of DATA command)

(User details edited as this is publicly archived)

I'm not sure I quite understand what's going on here. Chris indicated in email that this would affect other users whose email provider has set this kind of SPF record.

Can we make an educated guess as to what proportion of email providers set this kind of SPF?

How many messages do we never get to see? Is it a problem? Or a small enough number of users that we just don't worry about it?

Thanks

Sam

#757: Research and Design for TNv3

ed — Fri, 11 Jul 2014 13:36:54 GMT

R&D for TNv3

#671: Replace core Search module with Apache Solr

jim — Sat, 11 Jan 2014 21:11:14 GMT

Issue & background During work on #610, it was discovered that of a 1/4GB database dump for TN.org, ~80% (180Mb) of it was related to the Drupal 6 core Search module.

It's worth noting this was raised when we migrated the site to the Puffin server in March 2013, but it's generally the case that the core Search module does not scale easily beyond a few thousand nodes.

www.transitionnetwork.org has 23,803 nodes at time of writing -- this is probably approaching the sensible limit of the core module's capability.

Note also, any future D7 or D8 version of the site would also hugely benefit from using Solr, so the server config part is time well spent.

Proposed solution

Add the Apache Solr option to BOA, re-run the installer to get it installed and configured automatically.
Add the ApacheSolr module and any related required modules to the TN D6 makefile -- it's not clear if the 6.x-3.x branch or 6.x-1.x branch is the right choice at present.
Build a new platform containing these modules, migrate a clone of STG to it.
Enable the modules, configure them, disable core Search.
Create a feature that wraps up config for Solr and required modules. Add to Git, add reference to feature to makefile
Test, tweak, repeat 3 & 4 & 5 as needed.
Migrate PROD to the new plaform, enabled feature, index site.

This could be parked until D7/8 migration, or not... Ed's call.

#884: RE: http://news.transitionnetwork.org

paul — Thu, 03 Dec 2015 12:40:03 GMT

Hi Chris, All
Can you help me to reset my password for paulbooker for
news.transitionnetwork.org? I just tried to use the reset password form but
I never received an email and when I Iooked at the settings,php file for
the website (generated by Aegir) I couldn't see immediately where to find
the database.
I think I may have missed some recent updates to news.transitionnetwork.org
so urgently need to resolve this today.
Not sure how this has fallen of my radar, but, I just noticed that
news.transitionnetwork.org is no longer mentioned on the platform page on
Aegir so may have got into thinking that this site no longer exists.
http://news.transitionnetwork.org
https://tn.puffin.webarch.net/hosting/platforms
--
Paul Booker
Drupal Support for Websites and Linux Servers
Website: http://www.paulbooker.co.uk
Tel: +44 01922 861636

#821: Projects forms being hammered by Spam

ed — Wed, 07 Jan 2015 09:53:33 GMT

Projects forms being hammered by spammers. I got 24 in the last 45 minutes.

What to do?

Lock off to a certain type of user?

Adding Sam as owner to follow this up

#764: Policy decisions re-assessment on BOA and Drupal security updates

annesley — Tue, 22 Jul 2014 14:10:38 GMT

on-line meeting 5 / August @ 14:00 GMT: we are phasing out the current D6 / BOA system. the new system may not use either. The TN.org website is not attractive to high level hackers or DOS attacks.

what are the risks with cancelling all further Unix, BOA and Drupal updates completely that do not allow direct un-mitigated access to the backend via bad PHP code / SQL?

#746: New comment notifications not being sent to content owners.

sam — Tue, 24 Jun 2014 09:27:11 GMT

Hi Paul, Annesley, Chris

Ed hasn't been getting notifications for new comments.

"Please check if new comment notifications are being sent to content owners. I don’t think I am receiving email alerts for my blog posts."

I'll email Rob to see if he's getting any.

Could you investigate?

Thanks

Sam

#899: Managing security after Feb 24th, 2016.

paul — Thu, 28 Jan 2016 15:22:28 GMT

Hello,

Just did some research to check how we will manage security after Feb 24th, 2016.

A small group of vendors (approved by the security team) will provide patches for core and some of the most commonly used contributed modules, that are used on their client websites. Security patches will be put in the Git repo for the D6LTS project on Drupal.org, and will be announced in the issue queue. We will just need to monitor this issue queue and apply any security patches.

However, these vendors will not be supporting ALL contributed modules. Each of the vendors will be maintaining lists, and providing them to the Drupal Security Team so they know which issues to include them on. With this in mind shall we send a copy of our list of contributed modules to each of the vendor companies and ask them to provide us with a list of our modules that they are currently not supporting? We can then decide how we should support the modules that are not supported by the vendors.

Best, Paul

#887: Lot's of failed logins on conference15.transitionnetwork.org

sam — Fri, 04 Dec 2015 11:39:10 GMT

Hi all

Overnight I had 150 notifications of failed login attempts and subsequent IP address bans from the https://en-gb.wordpress.org/plugins/wordfence/ security plugin I installed.

It's coming from multiple IP addresses in multiple countries.

It seems like Wordfence is doing it's job and blocking IP's. I only mention it as I'm wondering if it could be related to the recent downtime.

Feel free to close this ticket, just thought it was worth sticking in here.

Thanks

Sam

#898: Fwd: Access to Drupal

ade — Tue, 26 Jan 2016 17:35:05 GMT

Hi Chris,
The web team at the development agency are requesting access to the
webserver so that they can look at the sites make up.
(Please see below)
Would you please set up an account so that they can get root read access?
I guess this would be done via FTP, but your thoughts greatly appreciated.
best regards
Ade
---------- Forwarded message ----------
From: Ainslie Beattie <ainsliebeattie@transitionnetwork.org>
Date: 26 January 2016 at 17:25
Subject: Fwd: Access to Drupal
To: Sam Rossiter <samrossiter@transitionnetwork.org>, Ade Stuart <
adestuart@transitionnetwork.org>, Yvonne Struthers <yvonne@thisisyoke.com>
Hey both, can you please action this urgently so that Yoke can have access.
Cheers
---------- Forwarded message ----------
From: "Yvonne Struthers" <yvonne@thisisyoke.com>
Date: 26 Jan 2016 10:58
Subject: Access to Drupal
To: <ainsliebeattie@transitionnetwork.org>
Cc:
Hi Ainslie,
Just a quick email as I'm out seeing a client today,but just to say,it
looks like you have only given us access to the database. What we need
please is admin access to the Drupal site and to the code base so that we
can get a sense of how it's all set up.
Thanks in advance!
Yvonne
Sent from my iPhone
--
Ade Stuart
Web Manager - Transition network
07595 331877
The Transition Network is a registered charity
address: 43 Fore St, Totnes, Devon, TQ9 5HN, UK
website: www.transitionnetwork.org
TN company no: 6135675 TN charity no: 1128675

#701: Emails & Telephone calls

paul — Tue, 18 Mar 2014 09:38:24 GMT

#590: Drupal performance improvements

jim — Fri, 06 Sep 2013 10:27:27 GMT

This ticket is to track the work and changes done within the Drupal sphere in relation to performance enhancements done since #585.

More information is needed and will come when ticket:586 New Relic Monitoring for BOA is completed.

I also note that many of these cleanup operations will also help make the move to D7 smoother and better.

Summary of actions and status

TODO

O) Stop making so many URL aliases for non-relevant pages, clean up url_alias table -- 1/4-1/2 hour, medium reward, only risk is that some already broken links might break... Per chat with Ed, only these will be removed (plus releated tweaks to Pathauto settings):

3,579 entries where src = node/%/feed
1,856 entries where src = user/%/contact
= 5,435 or ~11% of entries in url_alias

L) Review slow query log, explain queries, tweak as necessary/flag poorly behaving modules. 2-4 hours, high reward, low risk... Keep looking at the slow query log and adjust Drupal or find patches as necessary. ALSO related Reduce your server's resource usage by moving MySQL temporary directory to tmpfs... Have opened ticket for this: #591 for Chris.

Done

A) Remove spam taxonomy entries ~~1/2 hour, Low risk, low reward -- See item 8 below. A simple delete from taxo term table where length > 50 is worth doing IMHO, and nothing I saw that would be clobbered is not spam.~~

B) Try a Taxonomy Cleanup: 3 hours, Medium risk, medium reward -- style module to try to merge terms with the same names and clean up the link tables back to nodes. Further, we can remove any taxonomies or relations to certain CTs that don't really add value.

D) Review Views caching ~~1 hour, low risk, high reward -- Utilise Views Content Cache this was done a while back but I think -- done (task 12) in comment 21.~~

F) Force blocks caches to cached appropriately (and be rendered/included only as needed) 1-2 hours, medium reward, low risk -- BOA packages the Block Cache Alter, which makes sure Drupal only renders blocks when needed. Potential small but nice boost quickly in whole site. -- per comment 22, block caching is disabled by other modules so this will have to go on hold for now.

H) Remove CustomError? module all together 1/2 hour, low risk, low reward -- We should take out the PHP code from the 403 section of CustomError? and put it into a simple page entry. See comment 6 below as this has happened for 404s (which need no PHP). We can then remove the CustomError? module all together, saving lots of sessions. I would go ahead and do this but since the 403 page has various displays depending on user type, I wanted to raise it here as it *may* have side effects. Or not...

I) Re-enable block caching. 2-6 hours, high risk, high reward -- Per comment 24, a module (probably Content Access) is stopping Drupal caching blocks, which for some of them means a fair amount of pointless overhead. We need to somehow get around this and get blocks cached if possible. R&D mainly, perhaps with some hacking/patching - but I'd stop short of doing this if so.

K) Add & enable Views Lite Pager on big views. ~~1 hour, low risk, low reward -- Using this module stops a heavy count query on views with pagers -- recommended for large sites.~~

M) Take control of Cron, and maximise time pages are cached for. .25h, high reward, low risk -- Cron is wiping the page cache, so we need to install https://drupal.org/project/elysia_cron so we can clear the page less often, and run other things when we want and the site is quieter. Now need per minute resolution set to get the best, see comment 33 and 34 for more...

N) Replace Admin Menu 1.x with 3.x -- will happen when #590 occurs, marking complete here -- ~~5 mins, high reward, low risk -- done when #582 happens, could be the cause of some load spikes as it occasionally goes made and does 2000-5000 queries~~~~

#541: Documentation of the WordPress sites

chris — Wed, 01 May 2013 20:25:57 GMT

These pages have been created for the documentation of the wiki:WordPress sites running on wiki:PenguinServer:

So far they only have a listing on the plugins for each site.

Ideally they would document all the plugins, the theme and the steps that need to be taken to upgrade each site and also any other things that need documenting.

Laura is this something you might be able to help with? I'm happy doing some work on it but you know your way around these sites far better than anyone else.

#644: AWstats Nginx config breaks aegir

jim — Mon, 09 Dec 2013 16:46:05 GMT

Since the last update we've had a silent ngnix error that means http://tn.puffin.webarch.net was not available.

I restarted nginx and got:

[  ok  ] Stopping Nginx Server...:
[ .... ] Starting Nginx Server...:nginx: [emerg] "log_format" directive is not allowed here in /etc/nginx/nginx.conf:28

Which equates to the AWstats entry which is now commented out per:

# log for awstats
#log_format apache '$remote_addr - $remote_user [$time_local] "$request" '
#                   '$status $body_bytes_sent "$http_referer" '
#                   '"$http_user_agent"';
#access_log         /var/log/nginx/awstats.log apache;

I/we need access to aegir more than AWStats, so I've commented out the lines above and restarted nginx. Aegir is back and working well.

This ticket is to find the correct log_format for modern nginx versions and reinstate AWstats -- assigning to Chris as a low priority thing.

#758: * Advisory ID: DRUPAL-SA-CORE-2014-003

paul — Wed, 16 Jul 2014 21:55:29 GMT

View online: https://www.drupal.org/SA-CORE-2014-003

Advisory ID: DRUPAL-SA-CORE-2014-003
Project: Drupal core [1]
Version: 6.x, 7.x
Date: 2014-July-16
Security risk: Critical [2]
Exploitable from: Remote
Vulnerability: Multiple vulnerabilities

Multiple vulnerabilities were fixed in the supported Drupal core versions 6 and 7.

.... Denial of service with malicious HTTP Host header (Base system - Drupal 6 and 7 - Critical)

Drupal core's multisite feature dynamically determines which configuration file to use based on the HTTP Host header.

The HTTP Host header validation does not sufficiently check maliciously-crafted header values, thereby exposing a denial of service vulnerability.

.... Access bypass (File module - Drupal 7 - Critical)

The File module included in Drupal 7 core allows attaching files to pieces of content. The module doesn't sufficiently check permission to view the attached file when attaching a file that was previously uploaded. This could allow attackers to gain access to private files.

This vulnerability is mitigated by the fact that the attacker must have permission to create or edit content with a file field.

Note: The Drupal 6 FileField? [3] module is affected by a similar issue (see SA-CONTRIB-2014-071 - FileField? - Access bypass [4]) and requires an update to the current security release of Drupal 6 core in order for the fix released there to work correctly. However, Drupal 6 core itself is not directly affected.

.... Cross-site scripting (Form API option groups - Drupal 6 and 7 - Moderately critical)

A cross-site scripting vulnerability was found due to Drupal's form API failing to sanitize option group labels in select elements. This vulnerability affects Drupal 6 core directly, and likely affects Drupal 7 forms provided by contributed or custom modules.

This vulnerability is mitigated by the fact that it requires the "administer taxonomy" permission to exploit in Drupal 6 core, and there is no known exploit within Drupal 7 core itself.

.... Cross-site scripting (Ajax system - Drupal 7 - Moderately critical)

A reflected cross-site scripting vulnerability was found in certain forms containing a combination of an Ajax-enabled textfield (for example, an autocomplete field) and a file field.

This vulnerability is mitigated by the fact that an attacker can only trigger the attack in a limited set of circumstances, usually requiring custom or contributed modules.

/A CVE identifier [5] will be requested, and added upon issuance, in

accordance with Drupal Security Team processes./

Drupal core 6.x versions prior to 6.32.
Drupal core 7.x versions prior to 7.29.

Install the latest version:

If you use Drupal 6.x, upgrade to Drupal core 6.32. [6]
If you use Drupal 7.x, upgrade to Drupal core 7.29. [7]

Also see the Drupal core [8] project page.

The denial of service vulnerability using malicious HTTP Host headers was

reported by Régis Leroy [9].

The access bypass vulnerability in the File module was reported by Ivan

Ch [10].

The cross-site scripting vulnerability with Form API option groups was

reported by Károly Négyesi [11].

The cross-site scripting vulnerability in the Ajax system was reported by

mani22test [12].

The denial of service vulnerability using malicious HTTP Host headers was

fixed by Régis Leroy [13], and by Klaus Purer [14] of the Drupal Security Team.

The access bypass vulnerability in the File module was fixed by Nate Haug

[15] and Ivan Ch [16], and by Drupal Security Team members David Rothstein [17], Heine Deelstra [18] and David Snopek [19].

The cross-site scripting vulnerability with Form API option groups was

fixed by Greg Knaddison [20] of the Drupal Security Team.

The cross-site scripting vulnerability in the Ajax system was fixed by

Neil Drumm [21] of the Drupal Security Team.

The Drupal Security Team [22]

The Drupal security team can be reached at security at drupal.org or via the contact form at http://drupal.org/contact [23].

Learn more about the Drupal Security team and their policies [24], writing secure code for Drupal [25], and securing your site [26].

Follow the Drupal Security Team on Twitter at https://twitter.com/drupalsecurity [27]

_ Security-news mailing list Security-news@… Unsubscribe at https://lists.drupal.org/mailman/listinfo/security-news