i18n_subsites plugin: implement jinja2.ext.i18n support

this commit introduces optional support for translatable templates

README.rst

@@ -1,24 +1,24 @@
-i18n subsites plugin
+======================
-===================
+ I18N Sub-sites Plugin
+======================
 
-This plugin extends the translations functionality by creating i8n-ized sub-sites for the default site.
+This plugin extends the translations functionality by creating internationalized sub-sites for the default site. It is therefore redundant with the *\*_LANG_{SAVE_AS,URL}* variables, so it disables them to prevent conflicts.
-It is therefore redundant with the *\*_LANG_{SAVE_AS,URL}* variables, so it disables them to prevent conflicts.
 
 What it does
-------------
+============
 1. The *\*_LANG_URL* and *\*_LANG_SAVE_AS* variables are set to their normal counterparts (e.g. *ARTICLE_URL*) so they don't conflict with this scheme.
 2. While building the site for *DEFAULT_LANG* the translations of pages and articles are not generated, but their relations to the original content is kept via links to them.
 3. For each non-default language a "sub-site" with a modified config [#conf]_ is created [#run]_, linking the translations to the originals (if available). The configured language code is appended to the *OUTPUT_PATH* and *SITEURL* of each sub-site.
 
 If *HIDE_UNTRANSLATED_CONTENT* is True (default), content without a translation for a language is generated as hidden (for pages) or draft (for articles) for the corresponding language sub-site.
 
-.. [#conf] for each language a config override is given in the *I18N_SUBSITES* dictionary
+.. [#conf] For each language a config override is given in the *I18N_SUBSITES* dictionary.
-.. [#run] using a new *PELICAN_CLASS* instance and its ``run`` method, so each subsite could even have a different *PELICAN_CLASS* if specified in *I18N_SUBSITES* conf overrides.
+.. [#run] Using a new *PELICAN_CLASS* instance and its ``run`` method, so each sub-site could even have a different *PELICAN_CLASS* if specified in *I18N_SUBSITES* conf overrides.
 
 Setting it up
--------------
+=============
 
-For each extra used language code a language specific variables overrides dictionary must be given (but can be empty) in the *I18N_SUBSITES* dictionary::
+For each extra used language code, a language-specific variables overrides dictionary must be given (but can be empty) in the *I18N_SUBSITES* dictionary::
 
     PLUGINS = ['i18n_subsites', ...]
 
@@ -29,24 +29,39 @@ For each extra used language code a language specific variables overrides dictio
 	    }
 	}
 
-- The language code is the language identifier used in the *lang* metadata. It is appended to *OUTPUT_PATH* and *SITEURL* of each i18n sub-site.
+- The language code is the language identifier used in the *lang* metadata. It is appended to *OUTPUT_PATH* and *SITEURL* of each I18N sub-site.
-- The i18n-ized config overrides dictionary may specify configuration variable overrides, e.g. a different *LOCALE*, *SITENAME*, *TIMEZONE*, etc. 
+- The internationalized config overrides dictionary may specify configuration variable overrides — e.g. a different *LOCALE*, *SITENAME*, *TIMEZONE*, etc. However, it **must not** override *OUTPUT_PATH* and *SITEURL* as they are modified automatically by appending the language code.
-  However, it **must not** override *OUTPUT_PATH* and *SITEURL* as they are modified automatically by appending the language subpath.
-  Most importantly, a localized [#local]_ theme can be specified in *THEME*.
 
-.. [#local] It is convenient to add language buttons to your theme in addition to the translations links.
+Localizing templates
+--------------------
+
+Most importantly, this plugin can use localized templates for each sub-site. There are two approaches to having the templates localized:
+
+- You can set a different *THEME* override for each language in *I18N_SUBSITES*, e.g. by making a copy of a theme ``my_theme`` to ``my_theme_lang`` and then editing the templates in the new localized theme. This approach means you don't have to deal with gettext ``*.po`` files, but it is harder to maintain over time.
+- You use only one theme and localize the templates using the `jinja2.ext.i18n Jinja2 extension <http://jinja.pocoo.org/docs/templates/#i18n>`_. For a kickstart read this `guide <./localizing_using_jinja2.rst>`_.
+
+It may be convenient to add language buttons to your theme in addition to the translation links. These buttons could, for example, point to the *SITEURL* of each (sub-)site. For this reason the plugin adds these variables to the template context:
+
+extra_siteurls
+  A dictionary mapping languages to their *SITEURL*. The *DEFAULT_LANG* language of the current sub-site is not included, so this dictionary serves as a complement to current *DEFAULT_LANG* and *SITEURL*. This dictionary is useful for implementing global language buttons.
+main_lang
+  The language of the top-level site — the original *DEFAULT_LANG*
+main_siteurl
+  The *SITEURL* of the top-level site — the original *SITEURL*
 
 Usage notes
------------
+===========
 - It is **mandatory** to specify *lang* metadata for each article and page as *DEFAULT_LANG* is later changed for each sub-site.
-- As with the original translations functionality, *slug* metadata is used to group translations. It is therefore often
+- As with the original translations functionality, *slug* metadata is used to group translations. It is therefore often convenient to compensate for this by overriding the content URL (which defaults to slug) using the *URL* and *Save_as* metadata.
-  convenient to compensate for this by overriding the content url (which defaults to slug) using the *url* and *save_as* metadata.
 
 Future plans
-------------
+============
-- Instead of specifying a different theme for each language, the ``jinja2.ext.i18n`` extension could be used. 
+
-  This would require some gettext and babel infrastructure.
+- add a test suite
 
 Development
------------
+===========
-Please file issues, pull requests at https://github.com/smartass101/pelican-plugins
+
+- A demo and test site is in the ``gh-pages`` branch and can be seen at http://smartass101.github.io/pelican-plugins/
+
+..  LocalWords:  lang metadata

i18n_subsites.py

@@ -6,8 +6,11 @@ import os
 import six
 import logging
 from itertools import chain
+from collections import defaultdict
 
-from pelican import signals, Pelican
+import gettext
+
+from pelican import signals
 from pelican.contents import Page, Article
 
 from ._regenerate_context_helpers import regenerate_context_articles
@@ -17,6 +20,7 @@ from ._regenerate_context_helpers import regenerate_context_articles
 # Global vars
 _main_site_generated = False
 _main_site_lang = "en"
+_main_siteurl = ''
 logger = logging.getLogger(__name__)
 
 
@@ -27,10 +31,14 @@ def disable_lang_vars(pelican_obj):
     e.g. ARTICLE_LANG_URL = ARTICLE_URL
     They would conflict with this plugin otherwise
     """
+    global _main_site_lang, _main_siteurl
     s = pelican_obj.settings
     for content in ['ARTICLE', 'PAGE']:
         for meta in ['_URL', '_SAVE_AS']:
             s[content + '_LANG' + meta] = s[content + meta]
+    if not _main_site_generated:
+        _main_site_lang = s['DEFAULT_LANG']
+        _main_siteurl = s['SITEURL']
 
 
     
@@ -42,21 +50,20 @@ def create_lang_subsites(pelican_obj):
     and set DELETE_OUTPUT_DIRECTORY to False to prevent deleting output from previous runs
     Then generate the subsite using a PELICAN_CLASS instance and its run method.
     """
-    global _main_site_generated, _main_site_lang
+    global _main_site_generated
     if _main_site_generated:      # make sure this is only called once
         return
     else:
         _main_site_generated = True
 
     orig_settings = pelican_obj.settings
-    _main_site_lang = orig_settings['DEFAULT_LANG']
     for lang, overrides in orig_settings.get('I18N_SUBSITES', {}).items():
         settings = orig_settings.copy()
         settings.update(overrides)
-        settings['SITEURL'] = orig_settings['SITEURL'] + '/' + lang
+        settings['SITEURL'] = _main_siteurl + '/' + lang
         settings['OUTPUT_PATH'] = os.path.join(orig_settings['OUTPUT_PATH'], lang, '')
         settings['DEFAULT_LANG'] = lang   # to change what is perceived as translations
-        settings['DELETE_OUTPUT_DIRECTORY'] = False # prevent deletion of previous runs
+        settings['DELETE_OUTPUT_DIRECTORY'] = False  # prevent deletion of previous runs
 
         cls = settings['PELICAN_CLASS']
         if isinstance(cls, six.string_types):
@@ -131,8 +138,40 @@ def update_generator_contents(generator, *args):
             regenerate_context_articles(generator)
 
 
+
+def install_templates_translations(generator):
+    """Install gettext translations for current DEFAULT_LANG in the jinja2.Environment
+
+    if the 'jinja2.ext.i18n' jinja2 extension is enabled
+    adds some useful variables into the template context
+    """
+    generator.context['main_siteurl'] = _main_siteurl
+    generator.context['main_lang'] = _main_site_lang
+    extra_siteurls = { lang: _main_siteurl + '/' + lang for lang in generator.settings.get('I18N_SUBSITES', {}).keys() }
+    extra_siteurls[_main_site_lang] = _main_siteurl
+    extra_siteurls.pop(generator.settings['DEFAULT_LANG'])
+    generator.context['extra_siteurls'] = extra_siteurls
+    
+    if 'jinja2.ext.i18n' not in generator.settings['JINJA_EXTENSIONS']:
+        return
+    domain = generator.settings.get('I18N_GETTEXT_DOMAIN', 'messages')
+    localedir = generator.settings.get('I18N_GETTEXT_LOCALEDIR')
+    if localedir is None:
+        localedir = os.path.join(generator.theme, 'translations/')
+    languages = [generator.settings['DEFAULT_LANG']]
+    try:
+        translations = gettext.translation(domain, localedir, languages)
+    except (IOError, OSError):
+        logger.error("Cannot find translations for language '{}' in '{}' with domain '{}'. Installing NullTranslations.".format(languages[0], localedir, domain))
+        translations = gettext.NullTranslations()
+    newstyle = generator.settings.get('I18N_GETTEXT_NEWSTYLE', True)
+    generator.env.install_gettext_translations(translations, newstyle)    
+
+
+
 def register():
     signals.initialized.connect(disable_lang_vars)
+    signals.generator_init.connect(install_templates_translations)
     signals.article_generator_finalized.connect(update_generator_contents)
     signals.page_generator_finalized.connect(update_generator_contents)
     signals.finalized.connect(create_lang_subsites)

localizing_using_jinja2.rst

@@ -0,0 +1,46 @@
+-----------------------------
+Localizing themes with Jinja2
+-----------------------------
+
+1. Localize templates
+---------------------
+
+To enable the |ext| extension in your templates, you must add it to 
+*JINJA_EXTENSIONS* in your Pelican configuration::
+
+  JINJA_EXTENSIONS = ['jinja2.ext.i18n', ...]
+
+Then follow the `Jinja2 templating documentation for the I18N plugin <http://jinja.pocoo.org/docs/templates/#i18n>`_ to make your templates localizable. To enable `newstyle gettext calls <http://jinja.pocoo.org/docs/extensions/#newstyle-gettext>`_ the *I18N_GETTEXT_NEWSTYLE* config variable must be set to ``True`` (default).
+
+.. |ext| replace:: ``jinja2.ext.i18n``
+
+2. Specify translations location
+--------------------------------
+
+The |ext| extension uses the `Python gettext library <http://docs.python.org/library/gettext.html>`_ for translating strings.
+
+In your Pelican config you can give the path in which to look for translations in the *I18N_GETTEXT_LOCALEDIR* variable. If not given, it is assumed to be the ``translations`` subfolder in the top folder of the theme specified by *THEME*.
+
+The domain of the translations (the name of each translation file is ``domain.mo``) is controlled by the *I18N_GETTEXT_DOMAIN* config variable (defaults to ``messages``).
+
+Example
+.......
+
+With the following in your Pelican settings file::
+
+  I18N_GETTEXT_LOCALEDIR = 'some/path/'
+  I18N_GETTEXT_DOMAIN = 'my_domain'
+
+… the translation for language 'cz' will be expected to be in ``some/path/cz/LC_MESSAGES/my_domain.mo``
+
+3. Extract translatable strings and translate them
+--------------------------------------------------
+
+There are many ways to extract translatable strings and create ``gettext`` compatible translations. You can create the ``*.mo`` files yourself, or you can use some helper tool as described in `the Python gettext library tutorial <http://docs.python.org/library/gettext.html#internationalizing-your-programs-and-modules>`_.
+
+Recommended tool: babel
+.......................
+
+`Babel <http://babel.pocoo.org/>`_ makes it easy to extract translatable strings from the localized Jinja2 templates and assists with creating translations as documented in this `Jinja2-Babel tutorial <http://pythonhosted.org/Flask-Babel/#translating-applications>`_ [#flask]_.
+
+.. [#flask] Although the tutorial is focused on Flask-based web applications, the linked translation tutorial is not Flask-specific.