Manpages - Encode_Locale.3pm

Encode::Locale - Determine the locale encoding

use Encode::Locale; use Encode; $string = decode(locale => $bytes); $bytes = encode(locale => $string); if (-t) { binmode(STDIN, “:encoding(console_in)”); binmode(STDOUT, “:encoding(console_out)”); binmode(STDERR, “:encoding(console_out)”); } # Processing file names passed in as arguments my $uni_filename = decode(locale => $ARGV[0]); open(my $fh, “<”, encode(locale_fs => $uni_filename)) || die “Cant open $uni_filename: $!”; binmode($fh, “:encoding(locale)”); …

In many applications it’s wise to let Perl use Unicode for the strings it processes. Most of the interfaces Perl has to the outside world are still byte based. Programs therefore need to decode byte strings that enter the program from the outside and encode them again on the way out.

The POSIX locale system is used to specify both the language conventions requested by the user and the preferred character set to consume and output. The Encode::Locale module looks up the charset and encoding (called a CODESET in the locale jargon) and arranges for the Encode module to know this encoding under the name locale. It means bytes obtained from the environment can be converted to Unicode strings by calling Encode::encode(locale => $bytes) and converted back again with Encode::decode(locale => $string).

Where file systems interfaces pass file names in and out of the program we also need care. The trend is for operating systems to use a fixed file encoding that don’t actually depend on the locale; and this module determines the most appropriate encoding for file names. The Encode module will know this encoding under the name locale_fs. For traditional Unix systems this will be an alias to the same encoding as locale.

For programs running in a terminal window (called a Console on some systems) the locale encoding is usually a good choice for what to expect as input and output. Some systems allows us to query the encoding set for the terminal and Encode::Locale will do that if available and make these encodings known under the Encode aliases console_in and console_out. For systems where we can’t determine the terminal encoding these will be aliased as the same encoding as locale. The advice is to use console_in for input known to come from the terminal and console_out for output to the terminal.

In addition to arranging for various Encode aliases the following functions and variables are provided:

decode_argv( )

decode_argv( Encode::FB_CROAK )

This will decode the command line arguments to perl (the @ARGV array) in-place. The function will by default replace characters that can’t be decoded by \x{FFFD}, the Unicode replacement character. Any argument provided is passed as CHECK to underlying Encode::decode() call. Pass the value Encode::FB_CROAK to have the decoding croak if not all the command line arguments can be decoded. See Handling Malformed Data in Encode for details on other options for CHECK.

env( $uni_key )

env( $uni_key => $uni_value )

Interface to get/set environment variables. Returns the current value as a Unicode string. The $uni_key and $uni_value arguments are expected to be Unicode strings as well. Passing undef as $uni_value deletes the environment variable named $uni_key. The returned value will have the characters that can’t be decoded replaced by \x{FFFD}, the Unicode replacement character. There is no interface to request alternative CHECK behavior as for decode_argv(). If you need that you need to call encode/decode yourself. For example: my $key = Encode::encode(locale => $uni_key, Encode::FB_CROAK); my $uni_value = Encode::decode(locale => $ENV{$key}, Encode::FB_CROAK);

reinit( )

reinit( $encoding )

Reinitialize the encodings from the locale. You want to call this function if you changed anything in the environment that might influence the locale. This function will croak if the determined encoding isn’t recognized by the Encode module. With argument force $ENCODING_… variables to set to the given value.

$ENCODING_LOCALE

The encoding name determined to be suitable for the current locale. Encode know this encoding as locale.

$ENCODING_LOCALE_FS

The encoding name determined to be suitable for file system interfaces involving file names. Encode know this encoding as locale_fs.

$ENCODING_CONSOLE_IN

$ENCODING_CONSOLE_OUT

The encodings to be used for reading and writing output to the a console. Encode know these encodings as console_in and console_out.

This table summarizes the mapping of the encodings set up by the Encode::Locale module:

Encode | | | Alias | Windows | Mac OS X | POSIX ------------------------------–—+-------–— locale | ANSI | nl_langinfo | nl_langinfo locale_fs | ANSI | UTF-8 | nl_langinfo console_in | OEM | nl_langinfo | nl_langinfo console_out | OEM | nl_langinfo | nl_langinfo

I18N::Langinfo, Encode, Term::Encoding

Copyright 2010 Gisle Aas <gisle@aas.no>.

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself.