A list can have from three up to dozens of parameters. Some listmasters need to create a set of lists that have the same profile. In order to simplify the apprehension of these parameters, list families define a lists typology. Families provide a new level for defaults: in the past, defaults in Sympa were global and most sites using Sympa needed multiple defaults for different groups of lists. Moreover, families allow listmasters to delegate a part of configuration list to owners, in a controlled way according to family properties. Distribution will provide defaults families.
A family provides a model for all of its lists. It is specified by the following characteristics:
free (no constraint), controlled (a set of available values defined for these parameters) or fixed (the value for the parameter is imposed by the family). That prevents lists from diverging from the original and it allows list owner customizations in a controlled way;Here is a list of operations performed on a family:
Families can be defined at the robot level, at the site level or on the distribution level (where default families are provided). So, you have to create a sub directory named after the family's name in a families directory:
Examples:
/home/sympa/etc/families/my_family /home/sympa/etc/my_robot/families/my_family
In this directory, you must provide the following files:
config.tt2 (mandatory);param_constraint.conf (mandatory);edit_list.conf;automatic_lists_description.conf;message.footer.tt2,message.header.tt2,message.footer.mime.tt2,message.header.mime.tt2,info.tt2;This is a list creation template, this file is mandatory. It provides default values for parameters. This file is an almost complete list configuration, with a number of missing fields (such as owner email) to be replaced by data obtained at the time of family instantiation. It is easy to create new list templates by modifying existing ones. See List template files and Template file format.
Example:
subject [% subject %]
status [% status %]
[% IF topic %]
topics [% topic %]
[% END %]
visibility noconceal
send privateoreditorkey
web_archive
access public
subscribe open_notify
shared_doc
d_edit [% shared_edit %]
d_read [% shared_read %]
lang [% language %]
[% FOREACH o = owner %]
owner
email [% o.email %]
profile privileged
[% IF o.gecos -%]
gecos [% o.gecos %]
[% END %]
[% END %]
[% IF moderator %]
[% FOREACH m = moderator %]
editor
email [% m.email %]
[% END %]
[% END %]
[% IF sql %]
include_sql_query
db_type [% sql.type %]
host [% sql.host %]
user [% sql.user %]
passwd [% sql.pwd %]
db_name [% sql.name %]
sql_query [% sql.query %]
[% END %]
ttl 360
This file is mandatory. It defines constraints on parameters. There are three kinds of constraints:
free parameters: no constraint on these parameters, they are not written in the param_constraint.conf file.controlled parameters: these parameters must select their values in a set of available values indicated in the param_constraint.conf file.fixed parameters: these parameters must have the imposed value indicated in the param_constraint.conf file.The parameters constraints will be checked at every list loading.
WARNING: Some parameters cannot be constrained, they are: msg_topic.keywords (see msg-topic),owner_include.source_parameter (see owner_include) and editor_include.source_parameter (see editor_include). About digest parameter (see digest), only days can be constrained.
Example:
lang fr,us archive.period days,week,month visibility conceal,noconceal shared_doc.d_read public shared_doc.d_edit editor
This is an optional file. It defines which parameters/files are editable by owners. See List editing. If the family does not have this file, Sympa will look for the one defined on robot level, server site level or distribution level (this file already exists without family context).
Note that by default, the family_name parameter is not writable, you should not change this editing right.
This file is used if you want to let users create / access to automatic lists using the Sympa web interface. Please, see the documentation related to this functionnality.
You can parse several files at list creation, later used by the list. Here are the files you can parse:
message.footer.tt2,message.header.tt2,message.footer.mime.tt2,message.header.mime.tt2,info.tt2These files will be parsed using the list family data defined in the XML file. You can use the same data in these files as in the config.tt2 file.
For example, if you add to the family directory a file named info.tt2 containing the following code :
List [% listname %] home page [% description %]
If your XML file contain something like this for each list :
<listname>mylist</listname> <description>A loooong text describing the purpose of the list</description>
Then, once you have instantiated the family, each list will contain an info file with the following content:
List mylist home page A loooong text describing the purpose of the list
Message footers and headers are likely to contain TT2 code themselves (for example, to create unsubscription links at the bottom of the list messages).
In that case, you can use the capacity, offered by TT2, to define custom tag delimitors.
Here's an example of such usage. Let's say I want to create lists with a family, and I want that each list automatically adds an unsubscription URL at the bottom of each message. For this, I'll need to use a message.footer file in each list.
I add a file named message.footer.tt2 to the family directory. It contains the following code :
[% TAGS <+ +> -%] The subject of the list is "<+ subject +>", click here to unsubscribe : [% wwsympa_url %]/auto_signoff/[% listname %]/[% user.escaped_email %]
“Subject” corresponds to a tag in the XML file. Let's say it contains a short description of the list.
Once the family has been instantiated, each list directory will contain a message.footer file containing the following code :
The subject of the list is "create and share our passion of scrap cooking", click here to unsubscribe : [% wwsympa_url %]/auto_signoff/[% listname %]/[% user.escaped_email %]
Each time a message is sent to the list (provided you set the merge_feature parameter to on), this file will be parsed and allow to display the following text at the bottom of each message:
The subject of the list is "create and share our passion of scrap cooking", click here to unsubscribe : http://lists.domain.tld/auto_signoff/mylist/bob.mcbob%40domain.tld
Families provide a new level of customization for scenarios (see Authorization scenarios), templates for service messages (see Site template files) and templates for web pages (see Web template files). Sympa looks for these files in the following level order: list, family, robot, server site or distribution.
Example of custom hierarchy:
/home/sympa/etc/families/myfamily/mail_tt2/ /home/sympa/etc/families/myfamily/mail_tt2/bye.tt2 /home/sympa/etc/families/myfamily/mail_tt2/welcome.tt2
Instantiation allows to generate lists. You must provide an XML file made of list descriptions, the root element being family and which is only composed of list elements. List elements are described in section XML file format. Each list is described by the set of values for affectation list parameters.
Here is a sample command to instantiate a family:
sympa.pl --instantiate_family my_family --robot samplerobot --input_file /path/to/my_file.xml
This means lists that belong to family my_family will be created under the robot my_robot and these lists are described in the file my_file.xml. Sympa will split this file into several XML files describing lists. Each list XML file is put in each list directory.
–close_unknown option can be added to automatically close undefined lists during a new instantation
–quiet option can be added to skip the report printed to STDOUT
Example:
<?xml version="1.0" ?>
<family>
<list>
<listname>liste1</listname>
<subject>a list example</subject>
<description/>
<status>open</status>
<shared_edit>editor</shared_edit>
<shared_read>private</shared_read>
<language>fr</language>
<owner multiple="1">
<email>foo@cru.fr</email>
<gecos>C.R.U.</gecos>
</owner>
<owner multiple="1">
<email>foo@emnsp.fr</email>
</owner>
<owner_include multiple="1">
<source>my_file</source>
</owner_include>
<sql>
<type>oracle</type>
<host>sqlserv.admin.univ-x.fr</host>
<user>stdutilisateur</user>
<pwd>monsecret</pwd>
<name>les_etudiants</name>
<query>SELECT DISTINCT email FROM etudiant</query>
</sql>
</list>
<list>
<listname>liste2</listname>
<subject>a list example</subject>
<description/>
<status>open</status>
<shared_edit>editor</shared_edit>
<shared_read>private</shared_read>
<language>fr</language>
<owner multiple="1">
<email>foo@cru.fr</email>
<gecos>C.R.U.</gecos>
</owner>
<owner multiple="1">
<email>foo@enmsp.fr</email>
</owner>
<owner_include multiple="1">
<source>my_file</source>
</owner_include>
<sql>
<type>oracle</type>
<host>sqlserv.admin.univ-x.fr</host>
<user>stdutilisateur</user>
<pwd>monsecret</pwd>
<name>les_etudiants</name>
<query>SELECT DISTINCT email FROM etudiant</query>
</sql>
</list>
...
</family>
Each instantiation describes lists. Compared with the previous instantiation, there are three cases:
family_closed, or if the list is recovered, the list XML file from the previous instantiation is got back to go on as a list modification then.After list creation or modification, parameters constraints are checked:
fixed parameter: the value must be the one imposed;controlled parameter: the value must be one of the set of available values;free parameter: there is no checking.diagram
In case of modification (see diagram), allowed customizations can be preserved:
config_changes file, values can be collected in the old list configuration file, according to new family properties:fixed parameter: the value is not collected,controlled parameter: the value is collected only if constraints are respected,free parameter: the value is collected;Notes:
error_config and listmasters are notified. Then they will have to perform any necessary action in order to put the list in use.family_closed and owners are notified.
To modify a family, you have to edit family files manually. The modification will be effective while the next instanciation.
WARNING: The family modification must be done just before an instantiation. Otherwise, alive lists would not respect new family properties and they would be set in status error_config immediately.
Closes every list (installed under the indicated robot) of this family: list status is set to family_closed, aliases are removed and subscribers are removed from DB (a dump is created in the list directory to allow restoration of the list).
Here is a sample command to close a family:
sympa.pl --close_family my_family --robot samplerobot
Adds a list to the family without instantiating the whole family. The list is created as if it was created during an instantiation, under the indicated robot. The XML file describes the list and the root element is <list>. List elements are described in section List creation on command line with sympa.pl.
Here is a sample command to add a list to a family:
sympa.pl --add_list my_family --robot samplerobot --input_file /path/to/my_file.xml
Closes the list installed under the indicated robot: the list status is set to family_closed, aliases are removed and subscribers are removed from DB (a dump is created in the list directory to allow restoring the list).
Here is a sample command to close a list family (same as an orphan list):
sympa.pl --close_list my_list@samplerobot
Modifies a family list without instantiating the whole family. The list (installed under the indicated robot) is modified as if it was modified during an instantiation. The XML file describes the list and the root element is <list>. List elements are described in section List creation on command line with sympa.pl.
Here is a sample command to modify a list to a family:
sympa.pl --modify_list my_family --robot samplerobot --input_file /path/to/my_file.xml
According to file edit_list.conf, editing rights are controlled. See List editing. But in a family context, constraints parameters are added to editing right as it is summarized in this array:
array
Note: in order to preserve list customization for instantiation, every parameter modified (through the web interface) is indicated in the config_changes file.
Using a global family message.footer.tt2 file, you can add at the end of each message sent from a family list a global unsubscription link. Here's what you could put in such message.footer.tt2:
[% TAGS <+ +> -%] To stop receiving messages from <+ family_config.display +>, click on this link: [% wwsympa_url %]/family_signoff_request/<+ family_config.name +>/[% user.escaped_email %]
By clicking this link, the user will be redirected to the Sympa web interface where she will be informed that a confirmation message was just sent to her. If she clicks the confirmation link in this mesasge, she will be removed from all the past and future lists of this family.
Automatic list creation allows you to define those potential lists through family parameters, but they will not be created at once. The mailing list creation is triggered when Sympa receives a message addressed to this list.
To enable automatic list creation, you will have to:
To do so, you have to configure your MTA for it to add a custom header field to messages. The easiest way is to customize your aliases manager, so that mails for automatic lists are not delivered to the normal queue program, but to the familyqueue dedicated one. For example, you can decide that the name of those lists will start with the auto- pattern, so you can process them separately from other lists you are hosting.
familyqueue expects 2 arguments: the list name and family name (whereas the queue program only expects the list address).
Now let's start with a use case: we need to communicate to groups of co-workers, depending on their age and their occupation. We decide that, for example, if we need to write to all CTOs who are fifty years old, we will use the auto-cto.50@lists.domain.com mailing list. The occupation and age informations are stored in our LDAP directory (but of course we could use any Sympa data source: SQL, files...). We will create the age-occupation family.
First of all we configure our MTA to deliver mail to 'auto-*' to familyqueue for the age-occupation family. We'll also need to tell the MTA to accept mail for addresses that do not yet exist since by default postfix will reject mail for unknown local users.
/etc/postfix/main.cf
...
transport_maps = regexp:/etc/postfix/transport_regexp
local_recipient_maps = pcre:/etc/postfix/local_recipient_regexp unix:passwd.byname $alias_maps
/etc/postfix/transport_regexp
/^.*-owner\@lists\.domain\.com$/ sympabounce:
/^auto-.*\@lists\.domain\.com$/ sympafamily:
/^.*\@lists\.domain\.com$/ sympa:
/etc/postfix/local_recipient_regexp
/^.*-owner\@lists\.domain\.com$/ 1
/^auto-.*\@lists\.domain\.com$/ 1
/etc/postfix/master.cf
sympa unix - n n - - pipe
flags=R user=sympa argv=/home/sympa/bin/queue ${recipient}
sympabounce unix - n n - - pipe
flags=R user=sympa argv=/home/sympa/bin/bouncequeue ${user}
sympafamily unix - n n - - pipe
flags=R user=sympa argv=/home/sympa/bin/familyqueue ${user} age-occupation
A mail sent to auto-cto.50@lists.domain.com will be queued to the /home/sympa/spool/automatic spool, defined by the queueautomatic sympa.conf parameter (see queueautomatic). The mail will first be processed by an instance of the sympa.pl process dedicated to automatic list creation, then the mail will be sent to the newly created mailing list.
If you don't use postfix or don't want to dig in postfix alias management, you have an alternative solution for automatic listes management: sympa-milter.
This program is a contribution by Jose-Marcio Martins da Cruz.
What it does is checking all incoming mails and, if it recognizes a message to an automatic list, adds the relevant headers in it and places it in Sympa's automatic spool. It replaces familyqueue.
For all the doc, we assume you're using sendmail.
This is the procedure to make it work:
You can download the latest version at the following address: http://j-chkmail.ensmp.fr/sympa-milter/.
Once you have the archive, decompress it: tar xzvf sympa-milter-0.6.tgz.
You will need the libmilter library to build the sympa milter. This can be found in the development files of sendmail. You can install it, for example on a RedHat system, with the following command:
yum install sendmail-devel
Then install the program:
# cd sympa-milter-0.6/ # ./configure # make # make install
The default install directory is /usr/local/sympa-milter/ (you can change this value with the –prefix configure option).
The install process also adds a launcher into /etc/init.d/, named sympa-milter. You'll need to setup links to it under /etc/rc3.d. If you're using Fedora like Linux distributions, you can use /sbin/chkconfig to setup these links.
/sbin/chkconfig sympa-milter on
You must then set up the configuration file, sympa-milter.conf. You will find a sample configuration file inside /usr/local/sympa-milter/etc directory. This file contains two sections whose border are XML-like tags. Inside a section, a parameter is defined on a single line by the sequence:
parameters_name parameter_value
<general> and </general> tags is used to define, well general parameters, related to the program execution. It contains the following items:syslog will include a string like [ID 000000 local6.info] in each log line, allowing you to identify the log level and facility.automatic spool in which messages should be placed;/usr/local/sympa-milter/var/sympa-milter.pid);sympa, but changeable by a configure script option); this must be the same as the one running sympa;sympa, but changeable by a configure script option); this must be the same as the one running sympa;<families> and </families> tags is used to define the regular expressions which will allow sympa-milter to catch list creation messages. This section can contain an unlimited number of identically built lines, following this syntax:family recipient_regular_expression
You should use “plussed aliases” (at least with sendmail) to identify user existence more easily.
Here is an example of sympa-milter.conf, filled-up with default values :
# # Section general # <general> log_level 10 log_facility local6 log_severity yes socket inet:2030@localhost spool_dir /usr/local/sympa-milter/var pid_file /usr/local/sympa-milter/var/sympa-milter.pid run_as_user sympa run_as_group sympa </general> # # Section families # <families> # Syntax : # family recipient regular expression # joe ^joe+.*@one.domain.com toto ^bob+toto@other.domain.com best ^best.*@another.domain.com </families>
You can use any regular expression to define the addresses used by your family.
What you must do to make all the thingy to work is:
O InputMailFilters=sympa-milter Xsympa-milter, S=inet:2030@localhost, T=C:2m;S:20s;R:20s;E:5m
auto : /dev/null
or
auto : "some_file"
Reload your MTA config. All set!
We need to create the appropriate etc/families/age-occupation/config.tt2. All the magic comes from the TT2 language capabilities. We define on-the-fly the LDAP source, thanks to TT2 macros.
/home/sympa/etc/families/age-occupation/config.tt2
...
user_data_source include2
[%
occupations = {
cto = { title=>"chief technical officer", abbr=>"CHIEF TECH OFF" },
coo = { title=>"chief operating officer", abbr=>"CHIEF OPER OFF" },
cio = { title=>"chief information officer", abbr=>"CHIEF INFO OFF" },
}
nemes = listname.split('-');
THROW autofamily "SYNTAX ERROR: listname must begin with 'auto-' " IF (nemes.size != 2 || nemes.0 != 'auto');
tokens = nemes.1.split('\.');
THROW autofamily "SYNTAX ERROR: wrong listname syntax" IF (tokens.size != 2 || ! occupations.${tokens.0} || tokens.1 < 20 || tokens.1 > 99 );
age = tokens.1 div 10;
%]
custom_subject [[% occupations.${tokens.0}.abbr %] OF [% tokens.1 %]]
subject Every [% tokens.1 %] years old [% occupations.${tokens.0}.title %]
include_ldap_query
attrs mail
filter (&(objectClass=inetOrgPerson)(employeeType=[% occupations.${tokens.0}.abbr %])(personAge=[% age %]*))
name ldap
port 389
host ldap.domain.com
passwd ldap_passwd
suffix dc=domain,dc=com
timeout 30
user cn=root,dc=domain,dc=com
scope sub
select all
The main variable you get is the name of the current mailing list via the listname variable as used in the example above.
Now we need to enable automatic list creation in Sympa. To do so, we have to:
automatic_list_feature parameter to on and define who can create automatic lists via the automatic_list_creation (points to an automatic_list_creation scenario);queueautomatic sympa.conf parameter to the spool location where we want these messages to be stored (it has to be different from the /home/sympa/spool/msg spool).
You can make Sympa delete automatic lists that were created with zero list members; to do so, you should set the automatic_list_removal parameter to if_empty.
/home/sympa/etc/sympa.conf
...
automatic_list_feature on
automatic_list_creation public
queueautomatic /home/sympa/spool/automatic
automatic_list_removal if_empty
While writing your own automatic_list_creation scenarios, be aware that:
smtp and smime authentication methods in scenario rules (you cannot request the md5 challenge). Moreover, only the do_it and reject actions are available.Now you can send message to auto-cio.40 or auto-cto.50, and the lists will be created on the fly.
You will receive an 'unknown list' error if either the syntax is incorrect or the number of subscriber is zero.
While automatic lists save a lot of efforts to listmasters, they still remain hard to use for users. Most of the difficulty resides in the proper constitution of a list name. For example, if you use three fields to build your lists names, users will need to know, in order to use an automatic list:
Most users will never adopt this system as long as you have to manually compose the list email address. That's why most systems using automatic lists do it through a specialized user interface, using Thunderbird plugins or a dedicated web service.
The user-friendly automatic lists is there to let user create and find automatic lists using a point and click interface in the Sympa web GUI (though the classical approach - lists are created by a mail sending - still works).
Here are three screenshots describing the core functionnalities of the automatic lists in the Sympa web interface.
Once it is configured, users can select a new tab from the main Sympa tab panel. It displays a form. In this form, the user must select one value in each category. These categories describe the people the user wants to contact. The name of the automatic list created will be a concatenation of the values corresponding to the radio buttons selected.
Once the form is submitted, Sympa check whether the list exists already. If not, the list is created with the same mechanism described in the automatic lists feature. The user is then automatically redirected to the message composing space of the automatic list.
The list keeps existing after its creation. the user can therefore use its archives. Finding a list is just a matter of filling the first form and, once the message composing space is displayed, click on the “archives” link in the list menu.
To make the automatic lists available through the web interface, these families must be declared in sympa.conf (or robot.conf) using the automatic_list_families parameter.
The following values are expected for each family:
name: the name of the family to be used by Sympa. It is the actual family name (i.e. the name of the directory containing the family definition);display: a short name used in the web interfaceprefix: the prefix that will be used to tag the lists of this family (to be interpreted by the MTA to redirect messages to this list to the familyqueue program;prefix_separator: the prefix that will be inserted between the prefix and the string composed from informations taken from the list creation form.classes_separator: the character string that will separate each class value in the list name.family_owners_list: the name of a list whose users will be allowed to create automatic lists if you use the automatic_list_creation.family_owner scenario.In this doc, we suppose you have defined this parameter with the following value:
automatic_list_families name=fo:prefix=fo:display=Company contact lists:prefix_separator=+:classes_separator=-:family_owners_list=fo@dev-sympa.cru.fr;name=cgt:prefix=cgt:display=Company other contact lists:prefix_separator=+:classes_separator=-:family_owners_list=cgt@dev-sympa.cru.fr
Also, to make the automatic lists creation form available, you must change the value of the automatic_list_creation parameter to family_owner. This is a default scenario provided by Sympa that will allow autoamtic list creation to the follwing users:
automatic_list_families parameter for each family.Here is the code of this scenario:
title.gettext Restricted to people subscribed to the list of family owners. is_listmaster([sender]) smtp,dkim,md5,smime -> do_it is_subscriber([family->family_owners_list],[sender]) smtp,dkim,md5,smime -> do_it true() smtp,dkim,md5,smime -> reject,quiet
Once you've done that, restart the web server. You should, as listmaster, see two new tabs in the main Sympa menu:
Clicking on one of thses tabs will redirect you to the automatic lists creation form which, if you are following this doc step by step, should still be empty as our automatic lists families are not created yet.
To set up this functionnality, you need first to create at least an automatic lists family. You can define as many families as you wish.
A feature that can be usefull is the family unsubscription, if you wish.
On thing important when you create the config.tt2 of the families is the order in which the semantics of the data contained in the family name. If you want your lists name to look like ”<prefix>+<age>-<occupation>@domaine.tld”, you will need to make sure that in your config.tt2 you actually use the first item you find in the list name as the age and the second as the occupation. Otherwise, you're screwed.
This is a very important point to keep in mind when you create the automatic_lists_description.conf file in the section below: make sure that the order you define for each class will make them appear at the right place in the list name, according to what you defined in your config.tt2.
To display a form for a family, you need to add a new file to the family folder: automatic_lists_description.conf. This file contains the description of the fields used to:
Here is a summary of the structure of this file:
[class
name <short ascii name>
stamp <A short description of the class>
description <A user focused explanation of the semantics of the class>
order <An integer>
instances [order <An integer> --- value <short ascii value> --- tag <A human readable description of the value> (%%%)] (n times)
] (n times)
This file can contain only one type of parameter: class. A class defines a particular quality used to describe the population we want to subscribe to a list: Function, age class, location, etc. The class itself is described with several metadata:
name: The name used to describe this class internally. Its only use is to name the radio button fields in the list creation form.stamp: A short name used to make a title in the lists creation formdescription: A longer description of the class displayed below the title in the list creation form.order: This field contain an integer. It corresponds to the position this class will have intht e list creation form AND the position in which it will be used in the list name. I'll repeat just to make it clear: The order of a class corresponds to the position this class will have in the list name. It must then be chosen carefully and definitively before you put the automatic lists in production. If you change this order, the users won't be able to retrieve the lists created prior to the order change.instances: The instances are the values a class can actually take. These values are all defined in this field in the form of a character string1). The string must be structured as follows:order is followed by an integer. This integer only defined the position this value will hold in the lists creation form. It has no other usage.value is the actual value of the instance, the one that will be used in the list name. It must then only contain sequences compatible with the local part of an email address as defined by the RFC 822.tag is a short text that will be displayed in the character form in lieu of the actual instance value.</code>
Here is the form that would be displayed by this version of automatic_lists_description.conf:
If you submit this form, using the exact configuration given as example in this doc, you will create a list called fo+researcher-rennes-bastards@domain.tld.