SQL-Ledger Homepage

SQL-Ledger User Forum

Forum index page

Log in | Register

Back to the board
Thread view  Mix view  Order


07.04.2016, 19:52

Character BUG - Unicode? (General)


I think it is better to start a new thread - the last UTF-8 thread is a little bit confusing.

Today I have tried several things to solve my problems with the german special characters.

I set up a complete new vhost on my server and added a new postgre user to be exclude all possible things.

So I first tried to install 2.6.27 - everything was working like a charme. Dataset creation with Austrian-chart.sql worked well and saving and retreiving the german special characters worked as it is supposed to do.

Next I removed all files, dropped the database and started with 3.0.5 - some interesting things happened:
I had to iconv the Austrian-chart.sql from ISO-8859-1 to UTF-8 to get the sql to database - it did not matter whether to put "AddDefaultCharset On" in my vhost configuration. Also importing this file via command line gave me errors:
"ERROR: invalid byte sequence for encoding "UTF8": 0xfc" is only one of them.

BUT: importing my iconved sql works like it is supposed to do
AND: SQL Ledger is working - but I have to use "German (ISO-8859-1)" in user preferences to get my data written correct to the database and also receive the correct from my database. I also can change records an my german special characters do not transform to unreadable code.

Next i removed all files once more and tried with 3.2.0 - I also had to iconv my Austrian-chart.sql from ISO-8859-1 to UTF-8 to get it to the database.
BUT then: it does not matter which preferences I use - with "German (ISO-8859-1)" and "AddDefaultCharset On" I'm getting my special charactes stored correctly in my database, but I do only get unreadable code returned when I want to show this data in my web frontend.
With "German (UTF8)" and "AddDefaultCharste On" not even the german special characters in Webfrontend are displayed properly e.g. "Lizensiert fr" is display as "Lizensiert für". But I can store my special characters in my database, but do not get them back correctly.

When I add "AddDefaultCharset utf-8" to my vhost configuration and choose "German (UTF-8)" the webfrontend is displayed properly, but my german special characters are not stored correctly in the database and so they are not displayed properly in webfrontend when I search for this records.

I've spend some hours to investigate in this behaviour and was not able to solve my problem - I have to say, our company has focus on MySQL and PHP and some experience with UTF-8 in webprojects - but unfortunately not with Postgres and Perl.

It would be nice if our problem can be solved.

Feel free to ask me if you have further questions regarding our system environment.

Next week I think I will try to install SQL Ledger on another gentoo server to exclude that our server has some buggy configuration. But I have to say, that we don't have any problems on this machine with temporary PHP/MySQL projects in UTF-8

Thanks for reading and trying to help to solve our issue.


Andreas Fragner



08.04.2016, 04:54

@ hartundweich

Character BUG - Unicode?

Today I tried on another gentoo machine - installed Postgre database server and followed install instrucions for 3.2.0.
Does not matter if I add "AddDefaultCharset On" to vhost configuration or not... same result as described above...

Client: Windows 7 German 64Bit, Firefox, IE, Chrome - all Browsers the same result - is it concering Windows 7? Or ist perl or postgre?

Any hints welcome...



09.04.2016, 18:24

@ hartundweich

[SOLUTION] or not? Character BUG - Unicode?


today I have solved this issue - I was not able to stop thinking about this issue...
I googled around for PERL and UTF-8 in conjunction with POSTGRESQL - and found some important note about "pg_enable_utf8" in DBD:Pg that says, that this thing is getting deprecated...
First I tried setting this option to "1" and "-1" - but other characters than the desired have been displayed. So I tried to remove "pg_enable_utf8" from source code and voila - everything is working like a charm when I add "AddDefaultCharset On" to my VHOST configuration...

Mozilla Firefox says in all Frames that encoding is Unicode and all data sent to the database are stored as they should be stored and getting them back to webfrontend also works perfect - all my special characters are shown - HURRA :-)

I'm using DBD:Pg 3.4.2 and DBI 1.634.0 as these are the stable versions for my gentoo system.

But now all my special characters are missing in my generated PDFs... changing from pdflatex to xelatex does not help... maybe my postgres database is not UTF-8 as it is trying to say? If I want correct PDF documents I have to reinsert "pg_enable_utf8 => 0" again...

Back to the board
Thread view  Mix view  Order
1024 Postings in 338 Threads, 378 registered users, 142 users online (0 registered, 142 guests)
SQL-Ledger User Forum | Admin contact
powered by my little forum