Opened 13 years ago

Closed 13 years ago

#1581 closed enhancement (fixed)

Implement an EITFixUp function for missing character encoding

Reported by: Janne <janne-mythtv@…> Owned by: danielk
Priority: minor Milestone: 0.20
Component: dvb Version: head
Severity: medium Keywords:
Cc: Ticket locked: no

Description

Attached patch fixes special characters for german Pro7Sat1 multiplex (DVB-T Berlin).

The epg is encoded in some western encoding (maybe 8859-9) but is not declared. The correct implementation of ISO 6937 decoding in [9357] (#1466) breaks the special characters. QString::fromLatin1() used at least for the more frequently used characters (ä,ö,ü,ß) to work.

The patch is not ready to apply since the check for the these special transport and the correct encoding is hardcoded into SIParser::HandleEIT.

I first wanted to extend privateTypes for this check but the privateTypes is only NetworkID specific. I can probably use something like

privateTypes.FIXEncoding = TransportID << 4 & encoding

but this is limeted to only one transport per network and to the ISO 8859 character sets.

Are the special EITFixUP methods needed for all channels in the Network? I'm thinking about extending the privateTypes with the TransportID or creating a new struct for handling this.

Input how this should or can be done is welcome.

Attachments (5)

pro7_encoding_fix2.patch (5.5 KB) - added by Janne <janne-mythtv@…> 13 years ago.
preliminary patch
pro7_encoding_fix3.patch (5.5 KB) - added by Janne <janne-mythtv@…> 13 years ago.
attached wrong file while creating the ticket
pro7sat1_eitfix_permute_onid_tid.patch (415 bytes) - added by Janne <janne-mythtv@…> 13 years ago.
eit_fixup_use_tsid.patch (981 bytes) - added by Janne <janne-mythtv@…> 13 years ago.
eitfixup_onid_tsid.patch (522 bytes) - added by Janne <janne-mythtv@…> 13 years ago.

Download all attachments as: .zip

Change History (15)

Changed 13 years ago by Janne <janne-mythtv@…>

Attachment: pro7_encoding_fix2.patch added

preliminary patch

comment:1 Changed 13 years ago by danielk

Milestone: 0.20
Version: head

The proper place for this is in an EITFixUp function.

The plan is to not do the eit table/descriptor -> QString transform until just before we insert the eit into the DB. But even so it looks like this would be serviceid specific and we only decide on fixups on a network basis. So I think the proper thing to do is to check for this serviceid in the fix up function. However you can't do this yet, since the fixup function is still working on the QString and not the raw eit descriptor.

The upshot is that this fix has to wait until we've gotten a little further on the EIT rewrite. I'd say 3-4 weeks until now we'll be ready for the patch..

comment:2 Changed 13 years ago by Janne <janne-mythtv@…>

Summary: specify missing character encoding for dvb_decode_textImplement an EITFixUp function for missing character encoding

Thanks. I'll wait until then. I thought about an EITFixUp function (see my dev post). But as you stated it not reasonable atm.

This problem is TransportID specific. But it might make sense to make the EITFixUp functions serviceID specific since it exists multiplexes with services from different providers.

The patch works as it is for the Berlin DVB-T Sat.1 multiplex. For different regions or DVB-C and DVB-S the constants of networkID and transportID in the siparser.cpp part must be customized.

Changed 13 years ago by Janne <janne-mythtv@…>

Attachment: pro7_encoding_fix3.patch added

attached wrong file while creating the ticket

comment:3 Changed 13 years ago by danielk

Resolution: fixed
Status: newclosed

(In [9675]) Fixes #1581. Refrences #1035. Adds an EIT fixup for a Pro7Sat channel which doesn't properly encoded it's strings.

The patch was from Janne but wasn't applied earlier because it didn't fit into the old EIT processing very nicely, now that it does I've updated the patch and applied it.

Changed 13 years ago by Janne <janne-mythtv@…>

comment:4 Changed 13 years ago by Janne <janne-mythtv@…>

Resolution: fixed
Status: closedreopened

Orginal Network Id and transport Id are permuted in [9675]. Attached patch fixes it.

Changed 13 years ago by Janne <janne-mythtv@…>

Attachment: eit_fixup_use_tsid.patch added

comment:5 Changed 13 years ago by Janne <janne-mythtv@…>

Overlooked that the original implementation use only "original network id" and "service id". The Pro7Sat1 fixup is transport specific.

I don't want to add all services on this transport since ONID and SID do not identify a unique service.

The 64 bit mess might be unneccessary if the kFixSubtitle services are exclusively on their transports.

comment:6 Changed 13 years ago by danielk

(In [9682]) References #1581. Fixes inversion of networkid and serviceid for Pro7Sat1 eit fixup.

comment:7 Changed 13 years ago by danielk

Resolution: fixed
Status: reopenedclosed

(In [9683]) Closes #1581. Use UTF8->UTF16 conversion on strings from DB.

comment:8 Changed 13 years ago by danielk

Janne, If we need to key on the TransportID it shouldn't be a big deal to switch to a uint64_t key. We just have to make it optional so that we don't have to change all the existing fixup keyings.

Changed 13 years ago by Janne <janne-mythtv@…>

Attachment: eitfixup_onid_tsid.patch added

comment:9 Changed 13 years ago by Janne <janne-mythtv@…>

Resolution: fixed
Status: closedreopened

the last patch interprets the 16 least significant bits of the key also as transport id.

Simple appraoch that holds the key at uint. For a limited number of service that needs eit-fixup fuctions conficts between transport- and serviceid are unlikely.

comment:10 Changed 13 years ago by danielk

Resolution: fixed
Status: reopenedclosed

(In [9690]) Fixes #1581. Keys Pro7Sat1 fixup to transportid and networkid, rather than the traditonal networkid and serviceid match.

Note: See TracTickets for help on using tickets.