Groupware around mailing lists with Sympa

 

October 2000

Olivier Salaün / Serge Aumont

sympa-authors@cru.fr

 

 

In this article, we will present Sympa, a modern Mailing Lists management software, which provides groupware features.

 

1         Introduction

1.1      Why mailing lists are still a modern groupware tool ?

Historically, mailing lists have been a very important factor for the growing of internet usage.

We must not forget that ListservTM was the main bitnet service at the beginning of the 80s. Mailing lists have been the unique groupware tool for a long time !

 

Mailing lists  have decisive trumps :

-         e-mail is a universal service of the Internet. You can reasonably consider that each group member has email access.

-         unlike forum, web or news, email is a push technology.

-         unlike chat or videoconferencing, email is an asynchronous service : there is no need to make an appointment.

-         email address is a reasonable person identifiant (not as good as a Distinguish Name:), so mailing lists subscription process is an effective definition of the set of members of the group. In particular, any good mailing lists software provides a control on subscription by a list administrator.

 

Since groupware can’t ignore mailing lists  why not build a groupware architecture around mailing lists …

 

(In the following article ML will stand for mailing lists)

1.2      Why Sympa ?

Majordomo-like MLs are not enough for educational groupware. Classical ML provides too poor features. Integrating ML and other applications becomes difficult.

 

Sympa is a free software, running on UNIX system. It has been developed by the C.R.U. (the French Universities Network Team) to solve various problems with classic ML software such as full MIME support, enhanced security, high performances for large services, dynamic web interface with additional groupware features, dynamically defined list members.

 

2         Advanced services in Sympa

2.1      Full MIME support

Sympa recognises incoming complex MIME structures. When receiving mail commands in Multipart/alternative format, it will decode each part and looks for commands in text/plain parts. Sympa’s bounce analyser also recognises RFC1891 compliant message deliver status.

 

Sympa also provides a good MIME compliance with messages it sends, especially on the following points :

-         digest messages are enclosed in a multipart/digest MIME structure.

-         footers and headers are clean MIME parts

-         command reports use the correct Charset, conforming to current locale

2.2      Internationalisation

Early developments of Sympa already focussed on internationalisation. Every translation is out of the code, thereby making the translation job easier. Command reports have yet been translated to French, Spanish, German, Italian, Finish and Chinese. The web interface creates more constraints for translators since HTML pages have a more complex structure than a message body. The web interface has currently been translated to French and Spanish.

 

Sympa provides a hierarchical internationalisation that aims at using the more adapted language when communicating with users. Sympa has a default language setup (English) ; you may define one for your site ; lists have an official language ; users may choose their preferred language. Sympa will initialise the user’s preferred language with the official language of the first list he/she subscribes to.

 

2.3      Privilege definition using scenario

A scenario is an ordered list  of rules that determines if the requestor is allowed to perform the current command. Each rule consists of a condition part, a required authentication method (smtp, md5 or smime) and a resulting action. The condition part is evaluated with a set of environment variables ([listname],[sender],[header->From],…).

 

Example: controlling "Who can post a message to a list".

 

is_editor([sender],[list])                smtp -> do_it

the message comes from the editor

it is distributed anyway

!is_subscriber([sender],[listname])       smtp -> reject

Message from non-subscriber are rejected

match([header->Content-type],/multipart/) smtp -> editor_key

attachments => submit to editor

true()                                    smtp -> request_auth

Other messages need to be authentified

true()                                    md5 -> do_it

Other messages authentified are distributed

 

This scenario has the following behaviour :

-         reject messages from non-subscribers

-         submit multipart messages to list moderator

-         request confirmation before sending accepted messages

 

Every command in Sympa is controlled by scenarios. The listmaster can thereby change the behaviour of commands to anything that is expressible by a scenario.

3         Web Portal

Sympa includes a web interface to all ML services. It has the following properties :

-         Shared data :  all data are shared with the mail robot itself, so no additional management is needed for the web interface.

-         Single CGI : A single CGI provides different views of the ML service including a searchable lists directory, a list of subscriptions and a complete view of each list depending on the users privilege.

-         Admin : it includes administration features for list owners, lists editors and listmaster (such as bouncing subscribers management, list creation and configuration, etc)

-         Easy to customize : the HTML of the web interface has been separated from the code.

-          Persistence : for performances purpose, the CGI was written to be persistent in memory using FastCGI.

3.1      Web archives with access control

Sympa's web interface proposes archives of the ML and can also restricts access to these archives. The control applied to the archives may be chosen for each list and is based on ML-specific notions. Typical politics are public or private (for subscribers only) archives but may

be easily extended by listmaster (using scenarios). This is an essential feature to provide to private workgroup.

4         Shared web

Sympa associates a shared web space to each list. It proposes basic functions (create dir, upload file, rename). Why adding this feature in Sympa ?

To gain from the Sympa's authentication scheme. A directory/file of the shared web can thereby be restricted to a population (subscribers/owners of a list) for either read or write access. To define privileges in Sympa you select a scenario.

5         Security and access control

5.1      Mail security

Because they are based on email, MLs rely on messages From: header fields to identify people and give them corresponding privileges. This is insufficient because From: header fields are easy to forge. Sympa (like many other ML software) uses confirmation keys for verifying the requestor identity. The drawback of this method is that it makes commands heavier to use. Sympa’s scenarios are a way to define for the entire site or for a list who is allowed to perform each operation but also what is the acceptable authentication method.

 

For some lists usage, confirmation keys for verifying the requestor is not secure enough. A better way to identify people is to use digital signatures. Sympa is able to use S/MIME signature as an optional or mandatory authentication method for each operation.

 

Sympa can also distribute S/MIME encrypted messages. In this case,  the list has its own X509 certificate. A nice configuration is to require S/MIME signed message for subscription, so Sympa can store each subscriber's X509 certificate. The welcome message is automatically signed by Sympa, so each subscriber gets a copy of the list's certificate. Then any subscriber can send an encrypted message to the list,  Sympa decrypts the incoming message and crypt it for each recipient of the list.

5.2      Web security

Control access on web documents is commonly handled by the HTTP server. With Apache you define a user/password database (.htpasswd) and then access rules on documents. This was not adapted to Sympa that already manages a user data. Moreover it could not rely on platform-dependant systems.

 

Every web operation with Sympa is performed via a single CGI, providing the access control feature. This ensures a uniform and complete authentication scheme. User password are stored in the preference table of Sympa's database. Passwords may be reminded and changed from the web. Authentication persistency is performed using HTTP Cookies. Cookies only  carry identification information (this HTTP client is olivier.salaun@cru.fr), whereas privileges (based on scenario) are evaluated for each requested operation.

 

If you have a HTTPS server running, then you can configure it to ask for optional user authentication based on X509 certificates. If the user does not have a personal certificate, he/she will have to authenticate using a password in an encrypted session. If the server receives a user certificate, no password is required. Thereby usage of HTTPS in Sympa makes it stronger and more user-friendly.

 

6         Integrating Sympa in your Information System

6.1      External definition of list members

Subscribers are called this way because they subscribe to the list ; ML population evolving with subscriptions and unsubscriptions. Now imagine you know exactly who is supposed to be on your ML and you don't want strangers in it. A good example of this usage is an administrative ML with all university students.

 

Sympa will allow you to define a ML, with a list of subscribers dynamically extracted from your RDBMS or LDAP directory. For Sympa to use your database information, you need to define user data sources (mainly the SQL/LDAP query) in the ML configuration file. Various universities now creates systematically at least one ML for each educational unit. This implies thousand of ML, but of course all the student records are already in the scholarship directory so Sympa can extract it dynamically. No additional management when renewal of students happens.

 

Example of an LDAP inclusions :

 

Extracting email attribute for a selection

Extracting emails from a GroupOfMembers

include_ldap_query

    host ldap.cru.fr

    suffix dc=cru, dc=fr

    filter (objectclass=newpilotperson)

    attrs mail

    select first

include_ldap_query

    host ldap.tempere.edu.se

    suffix ou=students, o= tempere.edu.se

    filter (courses=BIO002)

    user sympa

    passwd xxxx

    attrs mgrprfc822mailmember

    select all

 

 

6.2      Sharing authentication

 

Another way to provide a nice integration of Sympa services in an existing information system is to share the authentication process with other application. The goals can either be to share the authentication form or only the user's password.

 

Sympa’s web interface uses a HTTP cookie that contains the user email after authentication. It can be recognized by some other application using a dedicated library. In this case, users can switch from Sympa to other services without further authentication process.

 

Nowadays the ideal way of managing user authentication is to store authentication user information in a central LDAP directory and all authentication operation should be able to use LDAP. We plan to include an auth_LDAP module in Sympa for April 2001 ; it will query LDAP directories for set Internet domains and manage passwords for others.

 

7         Conclusion

 

Sympa, distributed with pre-configured command scenarios and template files, is a scalable product. Its usage reflects this quality and we know Sympa is used by :

-         Almost all French universities and others in Italy, Germany, USA, Japan… They mainly benefit from its web interface (with both internet & intranet views), LDAP/SQL extraction features.

-         ISPs (Voila.fr, List Avenue) for its internal structure (RDBMS) and high delivery performances.

-         Companies because Sympa provides a secured extranet definition.

-         Non English-speaking users, finding in Sympa real internationalised product.

 

Sympa was first released in April 1997 and because ML are still up to date tool, this project is still under active development . We plan in future version several new features such as : 

-         User format reception option will allow users to receive messages in there prefered text format (as far as the initial message content type is multipart/alternative).

-         Pluggin filters to scan incoming messages with antiviral applications.

-         New LDAP usages for authentication, dynamic list owner definition and queries in scenario conditions.

-         Current virtual robot definition will be improved with a new concept in Sympa : list policies. Because Sympa allows a lot of different list usage, the list configuration is complex and may be difficult to master for list owners. In addition listmaster may loose control over lists, being misappropriated regarding the initial  project. The list policy will be a set of constraints on list configurations, defined by listmaster. This will ensure list type integrity and make owners configuration job easier. Typical list policies would be “newsletter”, “workgroup”, “discussion”,”webforum”,“hotline”, “administrative”.