June 14, 2010 - Fixing query troubles

This release fixes problems related to message sending preparation for some RDBMS.

June 07, 2010 - Fixing database creation problems

This release includes fixes related to database creation and DKIM.

June 03, 2010 - Fixing first problems in Sympa 6.1 beta.

This release includes the code necessary to unsubscription URL at the bottom of list messages.

June 03, 2010 - The latest stabilization version of Sympa 6.0 is out now!

This release includes several annoying bug fixes.

The CRU team is pleased to deliver Sympa 6.1 to the open source community. Sympa 6.1 is the result of more than one year of project activity and proposes several new interesting features.

[development by Julien Jourdan]

It is now possible to customize messages content with data specific to each subscriber.

The custom attributes are as parameters for TT2 tags in messages distributed. This adds notably the possibility to add authenticated unsubscribe URL to message footers. merge is controled by a list/robot parameter : merge_feature on|off

Sympa now supports DKIM for message diffusion and control. DKIM is a important tool for antispam but it will really become powerfull when most of the messages received in your mailboxes include a DKIM signature. That's why it is so important for MLM to support the DKIM standard. The DKIM features included in Sympa 6.1 were developped with the support of Internet2. Listmasters who want to learn more about DKIM and MLM should read the IETF draft.

* Incoming messages: Authorization scenarios can now use DKIM. Turning on the dkim_feature configuration parameter will provide a new authentication level to the authorization scenarios engine. Scenario evaluation for incoming messages with a valid DKIM signature (but no S/MIME signature) will be evaluated with authentication method dkim. So rules that use authentication method “smtp” will not match. * Outgoing messages: You may want to make Sympa sign outgoing messages. Almost every aspects of DKIM signature behavior can be customized via Sympa configuration parameters. Please check the DKIM parameters section for further details. Note that each parameter can also be set for a given virtual robot; and most of them are available as list parameter.

• dkim_feature
• dkim_add_signature_to
• dkim_signature_apply_on
• dkim_private_key_path
• dkim_selector
• dkim_signer_domain
• dkim_signer_identity
• dkim_header_list

See the documentation for details.

• dkim_feature
• dkim_signature_apply_on
• dkim_parameters
  • private_key_path
  • selector
  • header_list
  • signer_domain
  • signer_identity

[development by Julien Jourdan]

It is possible to exclude users from lists, event if they were subscribed using an external data source. Beware, that this feature is different from the blacklist. Exclusion makes sense when a list is based on external data sources, thanks to exclusion, a user can unsubscribe (or being removed by list owner) even though he should be included via an external data source. Users get into the exclusion table, through the standard unsubscribe / delete functions. They get off the exclusion table with the standard subscribe / add functions. Adding of the table 'exclusion_table'.

[development by Julien Jourdan]

The user can suspend her subscription. She is still registered as a subscriber but won't receive the mails.Subscription can be indefinite or have a scheduled expiration date. The subscription suspension can be managed for all subscriptions in a unique web interface.

complexWhich service include subscribers information about bounce

Most supported languages have seen their translation upgraded. Among them we can remark those languages who are approximately 100 % translated:

        o Polish (thanks to M. Woloszyn)
        o Japanese (thanks to S. Ikeda)
        o Russian (thanks to Chernysh)
        o German (thanks to M. Warkus)
        o French (eh...)
        
    In addition, M. Warkus did a great deal of work on Sympa typography: all the ponctuation signs are now included in the transaltion strings, so you can adapt the typography to whatever the best practices are in your language.

The format of the generic_sso paragraph in auth.conf was extended

Two new parameters :

• http_header_list : allows to explicitely list HTTP header fields that should be cached in sympa user_table.
• http_header_value_separator : user attributes may be multi-valued (including the user email address. This parameter defines the values separator; default is ';' (adapted for Shibboleth)

• spam_status message property is now evaluated from a scenario spam_status. This scenario accept ham, spam and unsure as result. This replace the antispam_header_name antispam_tag_header_spam_regexp
• Is is now possible to enable topics when the reception mode is 'mail', 'notice', 'not_me', 'txt', 'html' or 'urlize'. AuthCAS module used to be distributed with Sympa. Since Sympa 6, it is a standalone CPAN module. The module still has the same name, therefore old installed versions of the module need to be removed

A lot of bugs were fixed in this version, too. Check the release notes for details about bug fixing.

The latest stabilization version of Sympa 6.0 is out now!

This release includes many bug fixes, amongst which one that caused messages to be sent two times.

This release includes, along with several bug fixes, updated translations for the following languages:

• German (M. Warkus),
• French (Sympa authors),
• Japanese (S. Ikeda),
• Russian (Chernysh),
• Polish (M. Woloszyn).

Thanks to the translators for their work!

The CRU is proud to bring you the first release of Sympa version 6, along with the first release of the Sympa virtual machine!

So these are the new versions available in the Sympa world:

• 6.0 is the new stable version. That means that all bug fixes will now apply to both Sympa 6.0 and 6.1 versions; support for the version 5.4 will be restricted to drastic - security - issues and gradually cease, support for 5.3 and older versions is finished.
• 6.1 will be the new unstable version of Sympa. It is in alpha test right now and will be open for beta test soon.

Please note that all the latest Sympa features will be presented during the Fall 2009 Internet2 Member Meeting.

• We will introduce the latest features during a side meeting
• We will also be able to speak about our experience in using Sympa as a group manager in virtual organizations.

Feel free to contact us during this event. We are really looking forward to meet Sympa users from the other side of the ocean!

New bulk.pl daemon installed with Sympa. This daemon is dedicated to mail distribution and allows to parallelize this process on a single server as well as on multiple servers. The main bulk.pl processes will automatically fork new processes, depending on the amount of messages left in the database. You can control this behavior with the parameters bulk_fork_threshold, bulk_max_count,bulk_fork_threshold.

Messages spool used by the bulk.pl daemon is stored in the database, in two tables :

• bulkspool_table : one entry per message
• bulkmailer_table : subset of receipients for each message

New associated sympa.conf parameters :

• bulk_fork_threshold : minimum number of packets in database before the bulk forks to increase sending rate
• bulk_max_count : max number of bulks that will run on the same server
• bulk_lazytime : number of seconds a slave bulk will remain running without processing a message before it spontaneously dies
• bulk_wait_to_fork : number of seconds a master bulk waits between two packets number checks
• sympa_packet_priority : default priority set to a packet to be sent by the bulk (for future use)
• pidfile_bulk : location of the daemon PID file

You can trigger messages distribution day and time. This can be set on a per-list basis.

Storage of configuration in database is on its way! The colors used in the web interface are now stored in database. This is our proff of concept, opening the way towards a full storage of configuration and spools in database. These features will be important to make Sympa redundant. the first visible effect of this feature is the Sympa GUI color editor : you just have to edit colors and save them to make your Sympa server even more beautiful!

[Olivier Lumineau, CRU] Fresh new web CSS for the web interface. Here is the list of changes in the web interface :

• New css.tt2
• A new Sympa logo
• New navigation menu through list admin menu and server admin menu
• Miscellaneous modifications in templates (list_admin_menu.tt2 not used anymore)

WARNING : if you customize your own templates you may notice some mess in the page layout. In such a case compare your templates with the new distributed version and report any difference to your own files...

Replace storage of password with encryption by md5 fingerprint. This make remind password impossible. So now a one time ticket table is created ticket are sent by email as an authentication token. Ticket can be used for lost password, create account, moderation request . It should be generalized to all operation that need a email chalenge.

Want to create a new list that really looks like that other one you own since a year? Just click on the “clone list” button and bam! your list is created with the axact same configuration as the original, safe for the owners.

Four new languages are now available by default in Sympa:

• Bokmal (B. C. Aasgaard)
• Polish (L. Zygmanski)
• Finn (J.P. Paloposki)
• Vietnamese (C. Siddall)

Thanks to the translators for their efforts!

• New propertie in object message : spam_status . This feature is used in modindex (listing of message awaiting moderation) to show message tagged as spam. Messages suspected as spam are not included into messages sent to list editor. 4 new related parameters :
  • antispam_feature default off
  • antispam_tag_header_name default X-Spam-Status
  • antispam_tag_header_spam_regexp default ^\s*Yes
  • antispam_tag_header_ham_regexp default ^\s*No

• Message moderation : [Peter Hart] Added a new function for the moderation of messages. Upon rejection of messages, personnalised messages can be created, saved and used to give more information as to why these messages were refused by the moderator.
• New verify_netmask scenario condition. [Peter Hart] exemple : verify_netmask('1.12.123.00/24') (changes in intanet scenarii are just comment because we don't have access to the local netmask of installation).
• Extended the ldap_alias_manager.pl [Roland Hopferwieser] It used template to create LDAP entries. Also added a simulation mode for debug purpose.

In order to help you test the latest Sympa features, we built a VMware machine on which Sympa and a Dokuwiki farm are installed. They are set up to work together. That means that you will be able, with a simple click to create a wiki associated to the list you want. That work was done by Gael Merlin, of CRU.

So once your VM is installed, you will have a blank Sympa server, ready to use and fully set up. you just need to provide basic network information to put it into action.

You can download your Sympa vmware server and get additional informations on the Sympa web site.

Enjoy!

After 6 weeks of beta test, It is time to release a new beta version in which all the bug fixes we have heard of are now fixed.

This version also includes a completely updated German translation, thanks to the efforts of M. Warkus.

Older Sympa users will also note the vanishing of the “bin/etc” directory which contained all the default configuration and configuration samples. All these files and directories are now located in the new “default” directory.

Finally, a new configure parameter, ”--enable-fhs” will trigger different installation naming and location rules. If you add this option to configure, the different files used by Sympa will respect the FHS standards. If not, the directories and locations will remain the same as they were in previous versions of Sympa.

Not activating this option will be particularly useful for:

• people upgrading from a previous installation, which used custom scripts in which files and directory locations are critical
• anybody installing from the tar.gz distribution. Indeed, most of the files and directories being gathered in a central install directory, it is easy to inspect and customize them. It is also easy to completely remove a Sympa installation.

Activating this option will be particularly useful for Sympa packagers.

P.S.: Should the beta test keep on running smoothly, the stable release should be due at the end of August.

The CRU team is pleased to deliver Sympa V6 to the open source community. Sympa V6 is the result of more than one year of project activity and is a great improvement to Sympa earlier versions.

People who want to beta test Sympa 6 will register and give informations so that we know in which environment Sympa was tested . In return, they will get a dedicated help from the Sympa development team during the full beta test. Answering to help request for a beta tester will be our first priority during these seven weeks. This will probably be at the expense of classical support lists, such as sympa-users, so feel free to involve in collaborative support to fill this lack.

If you want to get involved, subscribe to the Sympa beta test mailing list. The subscriber review of this list is closed, safe to the list owners (the Sympa authors). This list is the official registration tool for beta testing. It can be a medium for general questions about the beta but, if you prefer, you can also send assistance requests directly to the sympa authors: sympa-authors@cru.fr.

Sympa comes with a new daemon that copes with low-level message distribution.The new bulk.pl daemon allows to parallelize mail sending across different processes and even on multiple servers. This is a major step towards Sympa clustering. Here are the nice features provided by the new bulk architecture already:

• message distribution recovery : distribution can now be stopped and restarted later without message duplication for subscribers.
• priorities : message for a list with a high priority are delivered with a very low latency even is a long distribution is started for a hugue list
• scheduled message processing : message distribution can be distributed at a specified hour (using a list parameter)

Related changes :

• you need to start a new bulk.pl process;
• once sympa.pl has processed a message, it is not delivered, but stored in the bulk spool;
• the bulk spool is stored in the database (shareable among different servers). You may need to tune the database in order to support large message.

  sympa.pl --test_database_message_buffer

  allow you to test the maximum size of messages that can be stored in your database.

The web interface has been renewed, thanks to Olivier Lumineau (CRU) work. Apart from the new look and feel, the main changes are the new navigation menus for both list administrators and listmasters features.

You can send HTML pages as messages to your list. this is simply done through the web interface. Just fill in the URL of the page you want to send and a message subject and bam! The page is sent. Please note that this message will be read by mail clients which are quite limited regarding web languages interpretation. So don't try sending pages in which scripts are essential for a good rendering.

Now password are stored as a fingerprint (by the past reversible encryption was used to be able to remind password). Reminding passwords is replaced by an identity token sent by email (thoses tokens can be used only one time and there validity it limited).

sympa.pl --md5_encode_password

Guillaume Rousse (IRISA) brought a lot of modifications were brought to the autotools usage in Sympa. This is still a work in progress, but should make the life of Sympa packagers quite easier once it is finished. This work has greatly simplified all Makefiles, turned into Makefile.am. Substitutions in perl modules are now performed at the “make” time, no more during “make install”

Here is the list of significant changes regarding the installation process:

• Default sympa configuration files, previously stored in PREFIX/bin/etc/ are now stored in PREFIX/share/default/ directory

• Most --with-xxdir configure options are now obsolete, replaced by equivalent standard --xxdir options.

• Most default installation directories have been change to adopt standard FHS-compliant locations. However, the distributed tar.gz still uses most of the previous locations to make upgrade process less awkward. If you run ”./configure ; make ; make install” with the tar.gz things will install as usually. If you run autoconf, it will define the set of target directories either the old way or the FHS way, depending on whether it finds an existing /etc/sympa.conf file.

• Removed obsolete sudo wrappers ; Sympa now includes C wrappers for both wwsympa.fcgi and sympa_soap_server.fcgi. Check the manual http://www.sympa.org/manual/index for more informations on that topic

• A new library is required to build Sympa : gettext-devel. This is necessary to compile the internationalization catalogues.

You can now create a list just by copying an existing list. The newly created list has the same configuration options and customization (message headers and footer for example). It has empty subscribers list, archives and shared directory.

• less_than. (Submitted by S. Amrani, gendarmerie nationale). It Allows to compare two values. Is true if the first one is less than the second one. Works with strings, too, if you like.
• verify_netmask. Exemple : verify_netmask('1.12.123.00/24') (changes in intranet scenarii are just commented because we don't have access to the local netmask of installation).

Moderators can compose rejection notification messages templates for automatic later use, on the web interface.

Sympa can recognize headers added by incoming spam message filters. Sympa does not answer automatically to those messages, they are submitted in special way to moderators both by mail or in the web interface for moderation. By default rejection is done quietly.

Now you may configure a list in order to send notification for new subscribers that are collected from an external datasource.

Setting some Sympa parameters is now possible using the web interface. Those parameters are stored in Sympa database. In version 6.0 only colors parameters can be changed using this method, but it settles the core functionalities used to fully transfer the Sympa configuration into its database, and its subsequent edition through the web interface.

Four new languages were recently added to the standard Sympa distribution. Many thanks to the translators:

• Bokmål (B.C. Aasgaard)
• Polish (L. Zygmanski)
• Finn (J.P. Paloposki)
• Vietnamese (C. Siddall)

In addition, Soji ikeda reactivated the legacy character support, as discussed in https://listes.cru.fr/sympa/arc/sympa-dev/2006-09/msg00029.html and its follow-ups. if the (new) sympa.conf parameter 'legacy_character_support_feature' value is set to 'on', the legacy character support is enabled.

Lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots and lots of bug fixes.

Just have a look at the NEWS file in the distribution to be convinced.

Pour présenter les nouveautés de la version 6 de Sympa, le CRU a organisé un webinar qui s'est déroulé le jeudi 14 mai 2009 à 10h30 pour les administrateurs de services de listes de diffusion,

Points abordés :

1. Présentation des nouvelles fonctionnalités de la version 6, ainsi que des évolutions de l'architecture,
2. Processus d'upgrade depuis les versions antérieures,
3. Le principe du beta test et du support aux beta-testeurs,
4. Où va Sympa ? Quelques axes de développement envisagés pour l'avenir.

We are pleased to announce that we are currently working on the latest features to be included in the next major version of Sympa: the 6.0.

Sympa 6 is an evolution of the software towards scalability and high availability. It will also offer a better ease of use and improved possibilities to adapt it to your particular needs and information system.

Shortly, the Sympa 6.0 will offer the following major features:

• Mail sending improvements: checkpoint recovery, scheduled sending, better priority handling...
• Making users life easier: personalized welcome page, good looking web interface, contextual help...
• For owners and moderator: reject message customization, list cloning...
• Easier packaging. The full autotools usage in Sympa was rewritten by Guillaume Rousse (INRIA) (thanks again Guillaume! ). This should greatly improve the life of Sympa packagers around here.
• As always: improved internationalization, thanks to the efforts all the fine people involved in Sympa translation.

Here is the scheduled timeline to this new Sympa version:

• Tuesday 5th May: release of Sympa 6.0 beta
• Tuesday 30th June: release of the stable version of Sympa 6.0

The most attentive readers will note that, for once, we spared a 7 weeks beta test period. That's because we felt a cruel lack of beta testing in the most recent releases of Sympa. This notably led to awkward bug reports a few days after the release. These were bugs on functionalities that were tested during several months on our own production system. Problem is: due to the high diversity of systems (OS, RDBMS, MTA, web servers...) likely to be used with Sympa, there are a lot of combinations we can't test. Early bug reports are the consequences of such combinations.

That's why we want to encourage people beta testing Sympa. We want you to install the beta version on production servers.

So this is our proposal: People who want to beta test Sympa 6 will register and give informations so that we know in which environment Sympa was tested . In return, they will get a dedicated help from the Sympa development team during the full beta test. Answering to help request for a beta tester will be our first priority during these seven weeks. This will probably be at the expense of classical support lists, such as sympa-users, so feel free to involve in collaborative support to fill this lack.

If you want to get involved, subscribe to the Sympa beta test mailing list. The subscriber review of this list is closed, safe to the list owners (the Sympa authors). This list is the official registration tool for beta testing. It can be a medium for general questions about the beta but, if you prefer, you can also send assistance requests directly to the sympa authors: sympa-authors@cru.fr.

Looking forward to see you in the beta!

Cheers,

The Sympa development team.

A few bug fixes in this version, which is mainly released to include the great translation work recently achieved. Thanks again to the translators!

Namely, the following languages have seen their translation improved:

• Catalan, thanks to J. Giralt
• English, thanks to M. Radford
• Estonian, thanks to U. Buhvestov
• Finn, thanks to J.P. Paloposki
• Japanese, thanks to S. Ikeda
• Russian, thanks to Chernysh
• Spanish, thanks to D. Viñar Ulriksen
• Vietnamese, thanks to C. Siddall

We also decided to make four new languages available by default in Sympa:

• Bokmal (B. C. Aasgaard)
• Polish (L. Zygmanski)
• Finn (J.P. Paloposki)
• Vietnamese (C. Siddall)

This release mainly fixes a bug making users miss an important warning when installing Sympa.

The sample SOAP client was broken ; it has been restored :

• try the SOAP client

This is a simple PHP SOAP client, but you can check the Sympa SOAP API definition to design more complex applications.

This release contains bug fixes and updated translations.

• Fix a bug regarding LDAP 2 level queries partial failures
• Fix a bug regarding blacklist edition. This is a potential security threat, so anybody using a Sympa server with blacklists enabled should consider upgrading shortly.

Thanks to all the contributors who submitted bug reports or even sent patches.

It also includes updated translations, amongst which:

• Finnish, thanks to J.P. Paloposki,
• English, thanks to A. Epstein,
• Nederlands, thanks to R. E. Sonneveld.

Thanks to all of them for their great work in Sympa internationalisation.

This release is aimed at the last bug fixes - including some security issues - for the 5.4 branch, amongst which :

• Sympa was not fully compliant to the RFC 2616, leading for example to possible unwanted list deletion by administrators using prefetching tools. This was fixed by replacing all the threatening GET requests by POST requests;
• Use of sprint() function for creating SQL queries lead to possible SQL injection through cookie manipulation;
• The use of files in /tmp lead to vulnerabilities.

Thanks to all the contributors who submitted bug reports or even sent patches.

It also includes updated translations, amongst which:

• Modern chinese, thanks to M. Smith,
• Vietnamese, thanks to C. Siddall,
• Russian, thanks to Chernysh,
• Bokmål, thanks to B.C. Aasgaard,
• Japanese, thanks to S. Ikeda,
• Magyar, thanks to H. Szabolcs,
• Finnish, thanks to J.P. Paloposki,
• Estonian, thanks to U. Buhvestov,
• Spanish, thanks to D. Magaña,
• German, thanks to S. Weber,
• Catalan, thanks to J. Deu-Pons.

Thanks to all of them for their great work in Sympa internationalisation.