Opened 16 years ago
Closed 16 years ago
#4519 closed defect (fixed)
MythWeather Environment Canada grabber broken
Reported by: | Owned by: | stuartm | |
---|---|---|---|
Priority: | minor | Milestone: | 0.21 |
Component: | mythweather | Version: | head |
Severity: | low | Keywords: | MythWeather, Environment Canada, grabber |
Cc: | stuartm | Ticket locked: | no |
Description
It looks like Environment Canada (http://www.weatheroffice.gc.ca) has changed their page layout. This has broken the ENVCAN grabber for MythWeather?.
However, it appears that the weather information is offered in an RSS feed now (i.e. http://www.weatheroffice.gc.ca/rss/city/ab-50_e.xml). I will update the grabber script and offer it as a patch shortly.
-- Joe Ripley vitaminjoe@…
Attachments (6)
Change History (18)
Changed 16 years ago by
Attachment: | 4519_diff.patch added |
---|
comment:1 Changed 16 years ago by
Attached patch should get things working again for the Environment Canada grabber. I haven't been able to test all possible renderings of the page (it'll take a couple days). I wouldn't commit this until I'm sure the bugs are worked out. I thought I'd post it now anyway, if anyone wants to test it.
I wanted to be able to use the RSS feed for parsing, however, the data on it is more abstract than the HTML data. Some of the RSS data is written in plain language, which could change, while the HTML data is numerical and probably won't. So, I had to stick with the HTML parser.
I'll keep an eye out for anything I missed and post patches if necessary.
-- Joe Ripley vitaminjoe@…
comment:2 Changed 16 years ago by
Owner: | changed from ldunning to stuartm |
---|---|
Status: | new → accepted |
Changed 16 years ago by
Attachment: | 4519_diff.2.patch added |
---|
Second version of patch to get EnvCan? grabber working. Fixes a few small details from the previous patch
comment:3 Changed 16 years ago by
Revised patch attached. This corrects a few small details missed by the first patch. Due to some inconsistency on Environment Canada's part, the 5 day forecast uses some kind of rotating id name. This causes some missing icons (sometimes!). This patch fixes that.
Still need to verify that wind gust speed is parsed correctly.
-- Joe Ripley vitaminjoe@…
comment:4 Changed 16 years ago by
Environment Canada sometimes doesn't report the high or low for a given day (not sure why). But this screws up the ordering of the data for MythWeather?. Patch is forthcoming...
-- Joe Ripley vitaminjoe@…
Changed 16 years ago by
Attachment: | 4519_diff.3.patch added |
---|
Makes use of RSS feed from Environment Canada for weather information
comment:5 Changed 16 years ago by
I've decided to use the RSS feed after all. It's layout has proven to be more consistent than the HTML layout. Also, each XML file is only about 4.5kB, which lessens the load on Environment Canada's servers as compared to the HTML files.
This patch adds the XML urls to ENVCAN-Stations.xml as well as a (nearly) completely re-written ENVCANParser.pm module. It also removes the need for the 'ENVCAN_icons' file.
Still need to do more testing, but this should work for most people.
-- Joe Ripley vitaminjoe@…
comment:6 Changed 16 years ago by
Status: | accepted → started |
---|
comment:7 follow-up: 10 Changed 16 years ago by
Oops. I just applied an earlier patch to ENVCANParser.pm. Sorry. Stuart, I will clean up the mess if you haven't started locally applying yet. Otherwise, fire this off to me?
Changed 16 years ago by
Attachment: | 4519_diff.4.patch added |
---|
Update ENVCAN grabber to use RSS feed from Environment Canada, also correct some whitespace issues in other modules
comment:9 Changed 16 years ago by
Just added the latest revision of the patch. I'm pretty confident that this one can be applied. I've reverted the earlier (larger) patch. The XML URLs aren't really necessary in ENVCAN-Stations.xml (neither are the HTML ones actually).
This patch does the following:
- Cleans up whitespace issues in all ENVCAN script components
- Modifies ENVCANParser.pm to parse XML RSS feed from Environment Canada
- Modifies envcan.pl to work with some minor interface changes in ENVCANParser.pm
Anything I've missed is probably trivial. This patch also takes care of issues that I brought up in ticket #4416. That one can be closed as invalid.
-- Joe Ripley vitaminjoe@…
comment:10 Changed 16 years ago by
Replying to anonymous:
Oops. I just applied an earlier patch to ENVCANParser.pm. Sorry. Stuart, I will clean up the mess if you haven't started locally applying yet. Otherwise, fire this off to me?
nigel: The patch I just submitted was diff'd against 15650, so it should apply cleanly to the changes that you applied in [15648].
-- Joe Ripley vitaminjoe@…
Changed 16 years ago by
Attachment: | 4519_diff.5.patch added |
---|
Fixed a couple very small details from the previous version of this patch
comment:11 Changed 16 years ago by
Sorry folks. I just noticed that the grabber was outputting 'mph' for wind speed when it wasn't necessary (i.e. the plugin does this already). Also, I had to correct for the instance where Environment Canada doesn't supply the 'high-2' data point.
Should be good to go now.
-- Joe Ripley vitaminjoe@…
Changed 16 years ago by
Attachment: | 4519_diff.6.patch added |
---|
Found another weather condition that wasn't being handled (overcast). Fixed in this version.
comment:12 Changed 16 years ago by
Resolution: | → fixed |
---|---|
Status: | started → closed |
Modified to handle changes in Environment Canada HTML layout, and fixes some whitespace issues