Opened 2 months ago

Last modified 2 months ago

#13517 new Bug Report - General

Reducing memory usage by mythfilldatabase to 1/6 of original usage

Reported by: dingo35 Owned by:
Priority: minor Milestone: needs_triage
Component: MythTV - Mythfilldatabase Version: Master Head
Severity: medium Keywords:
Cc: Ticket locked: no

Description

Mythfilldatabase uses QDomDocument to parse and store the XML data that is read; according to QDomDocument documentation this object is not meant to handle large XML files; QXmlStreamReader should be used in these situations.

This pull request replaces QDomDocument by QXmlStreamreader; loading a 222 MB XML file used 2,5GB on memory, after these changes it only uses 420MB, and speeds up the loading process by approx. 15% .

Pull request made on github: https://github.com/MythTV/mythtv/pull/189

Change History (1)

comment:1 Changed 2 months ago by dingo35

Testing can be done very easily:


mysqldump mythconver -u root >backup.sql
mythfilldatabase <your parameters> <your xml file>
mysqldump --skip-comments --skip-extended-insert mythconverg -u root >current.sql

//restore backup
mysql mythconverg -u root <backup.sql
<path-to-newly-compiled>mythfilldatabase <your parameters> <your xml file>
mysqldump --skip-comments --skip-extended-insert mythconverg -u root >new.sql

diff -Naur current.sql new.sql >output.diff

If you look at output.diff you should only see a few differences in timestamps and some changes in non-relevant tables...

Last edited 2 months ago by dingo35 (previous) (diff)
Note: See TracTickets for help on using tickets.