Australian EIT Fixups

[lexvictory@…]

Patch to add EIT fixups for Australian data, making it mostly usable as the only guide data source. Normalises titles/descriptions, adds in episode titles where available, repeats, teletext, ratings.

Also provides the correct genres based on the available Freeview info - was not done 100% before as it's not quite the same as the UK; only the main types are defined in nibble level 1. May need to be moved elsewhere?

I needed to copy a piece of code to allow the fixups to amend the ratings, as before the function handling updates was ignoring the ratings.

I have only tested this with Western Australia data, as I don't have access to any other states' data or network IDs.

[lexvictory@…]

Implements Australian EIT fixups & enables fixups to amend ratings

[dekarl@…]

If you don't know the network_id you can likely get around with other ids.

You can look the ids up at ​http://www.dvbservices.com/identifiers/original_network_id e.g.

0x2024 	Australian Digital Terrestrial Television	Australian Broadcasting Authority

that would be fix[ 8228U << 16 ] = thisnthat

network_ids for DTT in Australia are 0x3201 - 0x3300 with country code 36

You could filter by original_network_id + service_id with: fix [8228U << 16 | sid] = xxx Or you could filter by transport_id (useful over here as the companies have their own transport usually) with: fix [tid << 32 | 8228U << 16] = xxx

[lexvictory@…]

unfortunately each multiplex (network) has different requirements for the fixups, so I don't know how much that would help; only one of the fixups would be able to be applied.

[dekarl@…]

That sounds very similar to the way the multiplexes are setup over here. (Each TV network has their own multiplex aka. transport. Not to be confused with DVB networks.)

If the numbers in your patch are the numbers that identify each unique multiplex, then you can do as follows.

The common "all australian DVB-T" rules for the description matches like this

fix [ 8228U << 16 ] = EITFixUp::kFixAUDescription; // DVB-T Australia whole original network

and the special, per multiplex(transport), rules get matched by the combination of original_network_id/transport_id as follows

fix [  4115U << 32 | 8228U << 16 ] = EITFixUp::kFixAUSeven; // Seven transport
fix [ 12801U << 32 | 8228U << 16 ] = EITFixUp::kFixAUFreeview; // ABC transport
fix [ 12803U << 32 | 8228U << 16 ] = EITFixUp::kFixAUFreeview | EITFixUp::kFixAUNine; // Nine transport

while at it you could remove one of the AUStar lines as they are duplicates.

[Xander Victory <lexvictory@…>]

I think I understand what you're getting at, but should I be finding the 8228U number in mythtv's database somewhere?

The numbers I've used identify each TV network's multiplex and come from the networkid column in dtv_multiplex. Will that still work with << 32? The comment before the fixup array seems to say otherwise..

[dekarl@…]

I stand corrected, your patch appears to be fine. Australian broadcasters decided to reuse the "color C" network_id allocation for their original_network_id, which is a stupid idea. At least they documented their wrongdoing in the implementation guidelines at ​http://www.freetv.com.au/content_common/pg-engineering-guides.seo

original_network_id (ONID); where there is one ONID per transport stream and the actual value may correspond to the network_id

Lets hope the other color C countries don't make the same mistake or there will likely be collisions.

[lexvictory@…]

Luckily those IDs that start with 411 are able to be found in the dvbservices table you linked to, so not all will risk clash. I could add in the transport IDs from myth's db too, however based on what I can find on the internet, they're different in other states, so would only work for WA streams.

[slackerlinux85@…]

Im trying this out locally now without the programdata.cpp changes it seems to improve some channels but in brisbane the data still isnt perfect(Ten multiplex is the worst they flag cooking and fishing shows as movies :/)

heres the Queensland network ID's im using also the patch has some trailing whitespace issues

    // Queensland, Australia
    fix[ 4112U << 16]  = EITFixUp::kFixAUDescription | EITFixUp::kFixAUFreeview; // ABC Brisbane
    fix[ 4114U << 16]  = EITFixUp::kFixAUDescription | EITFixUp::kFixAUFreeview | EITFixUp::kFixAUNine;; // Nine Brisbane
    fix[ 4115U << 16]  = EITFixUp::kFixAUDescription | EITFixUp::kFixAUSeven; // Seven Brisbane
    fix[ 4116U << 16]  = EITFixUp::kFixAUDescription; // Ten Brisbane
    fix[ 12802U << 16] = EITFixUp::kFixAUDescription; // SBS Brisbane
    fix[ 12842U << 16] = EITFixUp::kFixAUDescription; // 31 Brisbane

[Stuart Auchterlonie <stuarta@…>]

In d5cc7a1c7ba43f3d0558773a7521694624a784ed/mythtv:

Add Australian EIT fixups.

This is a good first pass, any further improvements welcomed.

Fixes #10098

[Stuart Auchterlonie <stuarta@…>]

In 914442c91830af93fb90586eac1cc216b0e13a4d/mythtv:

Add Australian EIT fixups.

This is a good first pass, any further improvements welcomed.

Fixes #10098

[matt@…]

I would like to suggest the following changes which I have tested for a long time using a script to send recordings to a NAS. Other people have also been using this script on the mythtv forum and have not reported issues so these changes are tested.

The lines I use are: TVTITLE=echo "$TVTITLE" | sed 's/Movie Premiere: g' | sed 's/- Season Premiereg' | sed 's/Movie: g' | sed 's/:g' | sed 's/?g' | sed 's/,g'

TVSUBTITLE=echo "$TVSUBTITLE" | sed 's/\[RPT\] g' | sed 's/(R) g' | sed 's/ - g' | sed 's/(G) g' | sed 's/(AV) g' | sed 's/(M) g' | sed 's/(PG) g' | sed 's/\[CC\] g' | sed 's/(MA) g'

These variables in my script come directly from a user job from the backend passing the title and subtitle.

To decode these lines I'm sure you know sed but I'll give two examples...

The title in Australia from EIT can look like..

Movie Premiere: Shrek 11

Notice the space between the two as the first sed turns it into....

Skrek 11

Next example..

The Office- Season Premiere

after my script it turns to

The Office

Also the subtitle causes issues if the recording rules are set to detect the episode has already been recorded based on description and subtitle which is the default in mythtv. The station broadcasts a show more than once in a week and adds [RPT] to show it is a repeat. Mythtv sees it as different and records the show twice.

Not a complete list of issues, but after getting sick of filtering out the junk I switched to Shepherd for my guidedata so have not been updating my script to keep up with the junk that goes into the guide.

Hope the above helps. As a feature request, would it be a better idea to just add in the ability for users to add and remove sed commands from a script file? This way each country can instantly make changes as they see fit.

TitleFilter??.sh

SubtitleFilter??.sh

and mythtv runs these to process / filter the data as it is sent to the database?

[Stuart Auchterlonie]

Can you please open a new ticket rather than reopening a ticket which was closed 4 years ago. Feel free to reference this ticket in your new one.

Regards Stuart