Language identification and IT: addressing problems of linguistic diversity on a global scale

Statement of Responsibility:
Constable, Peter and Gary Simons
Series Issue:
2000-001
Date:
2000
Part Of Series:
SIL Electronic Working Papers 2000-001
Extent:
22 pages
Abstract:

Many processes used within information technology need to be customized to work for specific languages. For this purpose, systems of tags are needed to identify the language in which information is expressed. Various systems exist and are commonly used, but all of them cover only a minor portion of languages used in the world today, and technologies are being applied to an increasingly diverse range of languages that go well beyond those already covered by these systems. Furthermore, there are several other problems that limit these systems in their ability to cope with these expanding needs. This paper examines five specific problem areas in existing tagging systems for language identification and proposes a particular solution that covers all the world's languages while addressing all five problems.

Publication Status:
Published
Content Language:
Work Type:
Subject:
Computer programs
information technology (IT)
internationalization (I18N)
ISO 639
language identification
linguistic diversity
RFC 1766
web development
XML
Nature of Work:
Entry Number:
7861