You are here: Leader.IT>Portal Web>CrawlerLanguagesCall (20 Jun 2016, GuidoBrugnara)Edit Attach

Crawler Languages

This is a call for proposals to provide Apache2 configuration of Leader.IT Web portale (http://www.leader.it) with pages recognisable by crawlers for indexation of pages translated in different languages.

The portal is based on Foswiki and Apache 2.2 on Wheezy Debian server.
Foswiki uses the http Accept-Language header sent by the browser to provide the texts in the language set in the browser.

Unfortunately, search engines have a hard time identifying the pages translated into languages other than English, and therefore, you want to make alternative URLs available to allow crawlers to index the pages even if they do not use the Accept-Language header.

A possible solution, without modifying Foswiki, is to use Apache 2 modules to filter the URLs and to modify them by adding the Accept-Language header.

The changes are to be applied as follows (with relative URL "Portal" as example):

URL from crawler/browser	http header from crawler/browser	converted URL	updated header
http://www.leader.it/it/Portal		http://www.leader.it/Portal	Accept-Language=it
http://www.leader.it/en/Portal		http://www.leader.it/Portal	Accept-Language=en

In these cases, a "permanent redirect" (ERROR 301) will instead be applied.

URL from crawler/browser	http header from crawler/browser	Redirect 301 URL
http://www.leader.it/it/Portal	Accept-Language=it, ...	http://www.leader.it/Portal
http://www.leader.it/en/Portal	Accept-Language=en, ...	http://www.leader.it/Portal

The following links have to be added in the HEAD section to send the pages to the browser (example with URL related to "http://www.leader.it/Portal"):

<head>
   . . . . . .
   <link rel="alternate" href="/it/Portal" hreflang="it" />
   <link rel="alternate" href="/en/Portal" hreflang="en" />
   <link rel="alternate" href="/Portal" hreflang="x-default" />
</head>

Technical details

Conditions

The freelancer will use his own installation with Apache 2.2 for testing.
The rules have to work with HTTP and with HTTPS.
The freelancer will hand over the Apache 2 rules necessary to achieve the aforementioned conditions which the developer will then install on the portal for the final test.
The freelancer can submit an alternative solution to that which was suggested, provided that it resolves the problem and does not lead to changes to Foswiki.