close
Comments you submit will be routed for moderation. If you have an account, please log in first.
Modify

Opened 7 months ago

Closed 6 months ago

#492 closed bug (worksforme)

AmigaPL->UTF-8 for sending mails problem

Reported by: amiga@… Owned by:
Priority: normal Milestone: YAM 2.9p1
Component: charset handling Version: 2.9
Severity: major Keywords:
Cc: OS Platform:
Blocked By: Blocking:
Release Notes:

Description

I have found problem with proper handling encding AmigaPL->UTF-8 for sent mails.
I have set encodings AmigaPL for System (Config->First step->Charset AmigaPL) and for Write (Config->Default charset AmigaPL).
After sending mail, all national characters are displayed with negative question mark or others wird characters displayed on any system with any webmail or other e-mail client app.

Please, could You fix this with YAM2.9p1 please?
I had to write my own AREXX script to make proper characters conversion from AmigaPL to UTF-8 for sending mails. I'm useing this script from version YAM2.8 or maybe earlier, can't remember now.

My configuration (short): AmigaOS 3.9+BB1+BB2+patches, PPC 604e, 060.

Attachments (6)

amigapl_codeset_standard.png (49.0 KB) - added by amiga@… 7 months ago.
AmigaPL charset translation table
export_ODP__AmigaPL_test_e-mail.eml (3.4 KB) - added by amiga@… 7 months ago.
This is exported e-mail - received ansfer from my tablet
mail_from_YAM-Screenshot_2013-12-28-21-17-31.png (58.2 KB) - added by amiga@… 7 months ago.
screenshot from my tablet - received mail from YAM
reply_from_tablet.png (83.7 KB) - added by amiga@… 7 months ago.
screenshot of my reply from my tablet
AmigaPL2UTF-8.rexx (3.0 KB) - added by amiga@… 7 months ago.
ARexx script for annotate to make conversion from AmigaPL->UTF-8
Screenshot_2014-01-17-20-22-17.png (120.2 KB) - added by amiga@… 7 months ago.
New test of Polish AmigaPL charset for sending mails

Download all attachments as: .zip

Change History (25)

comment:1 follow-up: Changed 45 years ago by amiga@…

  • Status changed from pending to new

comment:2 Changed 45 years ago by amiga@…

  • Status changed from pending to new

comment:3 Changed 45 years ago by amiga@…

  • Status changed from pending to new

comment:1 follow-up: Changed 7 months ago by damato

  • Status changed from new to pending

To properly understand (we are not using polish charsets here) and identify the problem please provide an example email and a step-by-step explanation of how we can reproduce the issue on our own systems. In addition, please provide your own AmigaPL charset translation tables which you might be using with codesets.library.

In addition, please provide screenshots of the actual problem as well as screenshots of how it actually should be.

Changed 7 months ago by amiga@…

AmigaPL charset translation table

Changed 7 months ago by amiga@…

This is exported e-mail - received ansfer from my tablet

Changed 7 months ago by amiga@…

screenshot from my tablet - received mail from YAM

Changed 7 months ago by amiga@…

screenshot of my reply from my tablet

comment:2 in reply to: ↑ 1 Changed 7 months ago by anonymous

Replying to damato:

To properly understand (we are not using polish charsets here) and identify the problem please provide an example email and a step-by-step explanation of how we can reproduce the issue on our own systems. In addition, please provide your own AmigaPL charset translation tables which you might be using with codesets.library.

In addition, please provide screenshots of the actual problem as well as screenshots of how it actually should be.

I have provided attachment (amigapl_codeset_standard.png) with print screen of the AmigaPL charset translation tables from Wiki.
To reproduce problem You need to:

  1. Install and configure AmigaOS 3.9 BB2 with YAM2.9
  2. Install this file: ftp://amigaftp:amigaftpqaz@ftp.pekdar.net//OS3.9-update/BoingBag39-2-polski.lha
  3. Setup Locale preferences to "Polska" and save configuration.
  4. Setup Input preferences (Keboard maps) to "Polska - AmigaPL-USA" and save configuration.
  5. Setup YAM with Config -> Firs steps -> System settings -> Charset -> set AmigaPL.
  6. Setup YAM with Config -> Write -> Message body -> Default charset -> set AmigaPL.
  7. Save YAM configuration.
  8. Type New mail and write some Polish national characters, You can do this by pressing and hold down left or right Alt and typeing letters: azsxcnol

Big Polish national letters with left or right Shift button pressed + Alt.

  1. Send e-mail to self? f.e. as i did.
  2. Open received e-mail f.e. with any webmail under Windows, Linux, or any mobile system with IMAP client as i did on muy tablet (example: mail_from_YAM-Screenshot_2013-12-28-21-17-31.png).
  3. Done

My example reply for mail from YAM you can see on attachment reply_from_tablet.png.
I commented on this screenshot what happend and you can see how problem of character encoding/conversion looks like now and how should be.
I hope my example is good enough to explain problem?
If i could do anything more to help you understand the nature od problem, let me know. I'll do my best.

comment:3 Changed 7 months ago by damato

Thanks for the information. I will review the situation shortly. However, please also provide the Arexx script you are mentioning in your original bug description which you seem to use.

Changed 7 months ago by amiga@…

ARexx script for annotate to make conversion from AmigaPL->UTF-8

comment:4 Changed 7 months ago by amiga@…

Arexx script attached. I use Annotate editor to edit text and make conversion AmigaPL->UTF-8 with my script.

comment:5 Changed 7 months ago by damato

  • Milestone set to YAM 2.9p1
  • Priority changed from undecided to normal

comment:6 follow-up: Changed 7 months ago by tboeckel

Please try to set the external editor's charset to "AmigaPL" (Config window -> Misc -> External editor) and try to reproduce this issue without your conversion script. Your script should be obsolete with YAM 2.9, because all the necessary charset conversions are handled by YAM internally already now.

If the "AmigaPL" charset is required anywhere else you should adapt YAM's configuration accordingly, too.

Changed 7 months ago by amiga@…

New test of Polish AmigaPL charset for sending mails

comment:7 in reply to: ↑ 6 ; follow-up: Changed 7 months ago by amiga@…

Replying to tboeckel:

Please try to set the external editor's charset to "AmigaPL" (Config window -> Misc -> External editor) and try to reproduce this issue without your conversion script. Your script should be obsolete with YAM 2.9, because all the necessary charset conversions are handled by YAM internally already now.

If the "AmigaPL" charset is required anywhere else you should adapt YAM's configuration accordingly, too.

I did necessary changes as You post with (Config window -> Misc -> External editor) and did setup with AmigaPL. Just after I took test with a new mail and sent it to my second mail account.
I checked received mail from YAM on my tablet and you can see what i saw.
Problem still persist.

comment:8 in reply to: ↑ 7 ; follow-up: Changed 7 months ago by damato

I did necessary changes as You post with (Config window -> Misc -> External editor) and did setup with AmigaPL. Just after I took test with a new mail and sent it to my second mail account.
I checked received mail from YAM on my tablet and you can see what i saw.
Problem still persist.

So please verify again: Are you using still using your ARexx script for conversion or have you removed it? Because with 2.9 on you don't have to convert anything to UTF8 because all internal files are already in UTF8 now.

comment:9 Changed 7 months ago by damato

  • Status changed from new to pending

After a deep investigation of the problem you reported I finally found the following:

  • To me it really seems to be that you are having YAM incorrectly configured. After setting up YAM correctly I have no problems in converting between AmigaPL<>UTF8. So please make sure that you have the following settings in YAM:
    1. Config->First steps->Charset: set to 'AmigaPL'
    2. Config->Write->Default charset: set to 'UTF-8'
    3. Config->Miscellaneous->Display charset: set to 'UTF-8' (if you have a UTF-8 capable editor, or to 'AmigaPL' otherwise)
    4. Make sure that you are NOT using your ARexx script anymore. With YAM 2.9 that script is obsolete!
  • While investigating your report, I identified a minor bug in codesets.library regarding the AmigaPL codeset (but unrelated to your reported issue): the 0xc484 (utf-8) character wasn't correctly converted (it was set to plain 'A') as it was missing in the conversion routines. In the meantime I fixed that, so that in an upcoming codesets.library update the AmigaPL<>UTF8 conversion routines should include all the special characters AmigaPL supports.

As said above, I really believe that the issue reported here is simply a misunderstanding and misconfiguration of YAM because with YAM 2.9 many UTF-8 related things changed and YAM is now perfectly able to send UTF-8 encoded emails itself. So please retry again by following my instructions above for setting up YAM correctly. Then please try to reproduce the problem again and reporting back if the issue is now fixed or if you are still having issues.

comment:10 in reply to: ↑ 8 ; follow-up: Changed 6 months ago by amiga@…

Replying to damato:

I did necessary changes as You post with (Config window -> Misc -> External editor) and did setup with AmigaPL. Just after I took test with a new mail and sent it to my second mail account.
I checked received mail from YAM on my tablet and you can see what i saw.
Problem still persist.

So please verify again: Are you using still using your ARexx script for conversion or have you removed it? Because with 2.9 on you don't have to convert anything to UTF8 because all internal files are already in UTF8 now.

I did and i did not use my ARexx script for conversion.

comment:11 in reply to: ↑ 10 ; follow-up: Changed 6 months ago by damato

  • Status changed from new to pending

Replying to amiga@…:

I did and i did not use my ARexx script for conversion.

Is that all information you can provide? Please have a look at my latest comment where I gave instructions how you are supposed to setup YAM so that it should correctly send UTF-8 encoded mails.

comment:12 in reply to: ↑ 11 Changed 6 months ago by amiga@…

Replying to damato:

Replying to amiga@…:

I did and i did not use my ARexx script for conversion.

Is that all information you can provide? Please have a look at my latest comment where I gave instructions how you are supposed to setup YAM so that it should correctly send UTF-8 encoded mails.

Of course not. During your answer in the same time i wrote my answer for last comment. I just want to not leave your question without answer about Arexx conversion. I'm not fast as you with english ;-)

comment:13 Changed 6 months ago by damato

  • Status changed from new to pending

Any news on that item? I thought I give clear instructions (see https://yam.ch/ticket/492#comment:9) on how to fix the problem which I believe is a simple misconfiguration. Do my instructions solve the problems related to AmigaPL for you?

comment:14 follow-ups: Changed 6 months ago by Pekdar <amiga@…>

Something is wrong. I can not send reply. The two times I filled out the form with my reply to comment:9 and 13 and nothing show up?
I always saw the answer "Comments you submit will be routed for moderation" ... and nothing.

What's going on?

comment:15 in reply to: ↑ 14 Changed 6 months ago by tboeckel

Replying to Pekdar <amiga@…>:

What's going on?

That happens when you are not logged in. Such comments must be approved by Jens or me before they become visible for everybody. Very obviously you have no account on yam.ch yet.

comment:16 in reply to: ↑ 14 Changed 6 months ago by damato

  • Resolution set to worksforme
  • Status changed from pending to closed

Replying to Pekdar <amiga@…>:

Something is wrong. I can not send reply. The two times I filled out the form with my reply to comment:9 and 13 and nothing show up?
I always saw the answer "Comments you submit will be routed for moderation" ... and nothing.

What's going on?

I actually checked the spam filter log files of this webpage and your replies were flagged as being SPAM because you weren't logged in and the spam probability of your answer was >80% for that particular reply. So please generate yourself a login for this web page and log in before you post anything.

However, here is an excerpt of the message you replied recently:

OK I'll write reply again (but much shorter) for your comment:9

I did all steps as You describe and now I have working conversion AmigaPL->UTF-8 with sent mails!
Thank You very much for help!

I discover that problem is only with BIG letter 'A' with polish character as You describe before.

Ok, so I guess the problem is finally sovled now. In the end it was a misunderstanding on how to setup YAM to write UTF-8 emails and where you should configure AmigaPL and where UTF-8. But this should be clear now.

In addition, if you install the latest released 6.17 version of codesets.library the problem with the uppercase 'A' should be solved as well.

Add Comment

Modify Ticket

Action
as closed .
The resolution will be deleted. Next status will be 'reopened'.
Author


E-mail address and user name can be saved in the Preferences.

 
Note: See TracTickets for help on using tickets.

This list contains all users that will be notified about changes made to this ticket.

These roles will be notified: Reporter, Owner, Subscriber

  • Pekdar(Reporter, Participant)