13 Change Notification Tools

We are going to discuss different tools that can notify you in different ways and in different circumstances when some specific thing has changed on the Web: email alerts, page monitoring software, and RSS feed-manipulation software.

Class held on 10/26/2009. (student notes; possible questions).

Class structure

  1. Go through “At beginning of class” information
  2. I'll lecture for a bit (no slides today).
  3. Work on exercises.

At beginning of class

On your own

  1. FYI, I added a scanned copy of the diagram I created for the real-time information class.
  2. Read the current to-do list on the course home page.
  3. No grades (too much other prep going on)
    • I am about 1/4 the way through the status reports. I'm working on them, I promise.
    • I haven't graded any blogs in a very long time.

My notes

change-notification.jpg

Monitoring changes

  1. Email alert service
    • Monitor entire site
    • These are set up by the Web site and you subscribe to them
    • No false positives
    • Sometimes you want email (cell phone! or even Messenger)
  2. Page monitors
    • Monitor specific pages (but not sites)
    • Lots of false positives unless keyword based
  3. RSS feeds
    • Problem: False positives
      • Unless keyword based or filtered somehow
    • Focused RSS feed — If you’re lucky, there is a keyword-based, or specific-topic defined, RSS feed available for a site you can subscribe to.
      • Specific sites (findable in all the usual ways)
      • Dapps at Dapper.net
      • Pipes at Yahoo Pipes
    • General RSS feed: If there's simply a general RSS feed (such as "Yahoo breaking news"), then you should run that feed through a keyword tool:
    • The following are useful if there's no RSS feed available on a page but you would like to set one up:
      • FeedYes: I would try this first since it's the easiest to use when setting up a feed.
      • Feed43: This is more powerful but more difficult to use.
      • Dapper: This is another powerful tool.
  4. Why not just use RSS
    • Some sites don't have RSS feeds
      • So use site-based email alerts
      • Or use a tool to make an RSS feed
    • Some information isn't site based
      • So use search-based email alerts
    • Some information is too fine-grained to be covered by RSS feeds
      • So use page monitors

Email alerts

Finding email alerts

  1. Search for email alerts
    • Query: "email alerts" OR "e-mail alerts" OR "email alert" OR "e-mail alert"
    • Google results (189 million in 2009) (77.2 million in 2008) (60.2 million in 2007)
    • Yahoo results (464 million in 2009) (392 million in 2008)
  2. More specific search for email alerts
    • Query: inurl:mail OR inurl:alert "email alerts" OR "e-mail alerts" OR "email alert" OR "e-mail alert"
    • Google results (88,100 in 2009) (115,000 in 2008)
    • Yahoo results (280,000 in 2009) (242,000 in 2008)
  3. Science email alerts
    • Query: science "email alerts" OR "e-mail alerts" OR "email alert" OR "e-mail alert"
      • Google results (38.6 million in 2009) (17.1 million in 2008) (2.34 million in 2007)
      • Yahoo results (72.3 million in 2009) (69.9 million in 2008)
    • INURL query
  4. Copper email alerts
    • Query: copper "email alerts" OR "e-mail alerts" OR "email alert" OR "e-mail alert"
    • INURL query
  5. So, think about how you might apply this both to a company you are interested in or an industry you are interested in

General email alert services

  1. Yahoo Alerts
    • Some types of alerts
    • Be sure to look over the whole list of categories of alerts.
  2. Google Alerts (help)
    • All of this is based on submitting queries
      • Once a day
      • Once a week
      • "As it happens"
    • Broad-ranging alerts
      • Web & comprehensive alerts
    • More specific
      • Keyword-based alerts for news, blogs, video and groups
    • Can receive as email or as an RSS feed

Page monitoring software

Overview

Page Monitors were the next big thing five years ago. It is a program or web based program that you download. Each day (or whatever time period you want to set) it downloads the webpage, and if it's different it will send you an email. Some tell you what has changed while others just tell you that it has changed.

At first, you might not be that impressed with page monitors. But after realizing that it can be used for a lot more than news, it can be quite a useful tool. WatchThatPage.com is the best free site.

WatchThatPage has a limit of 250 characters for the URL. Also, shortened URLs (from tinyurl.com or bit.ly) do not work. To get around these problems, use TrackEngine, where neither of these problems exist.

Web-based

  • WatchThatPage
    • Free (for any number of pages), or $20/year for priority service
    • Can highlight changes in pages
    • Changes sent in an email
    • Keyword matching
    • This site doesn't appear to be updated any more (4+ years)
  • TrackEngine
    • Free for 5 bookmarks, or $20/year for 10 pages, or $59/year for 50 pages
    • Highlights new content in HTML email
    • Monitors changes daily
    • Does do keyword matching
    • This site hasn't been worked on for 7+ years
  • Other possible sites: InfoMinder, ChangeDetect, Trackle

Windows software

Feed creation software

Overview

Make a feed

From other feeds
From a page
  • Dapper
  • FeedYes
  • Feed43
    • Feed43 is a little bit more complicated. You have to find the actual html within the source code of the page.
      • Define Extraction Rules – By finding the specific places (within the code) of the information that you’re looking to have monitored by the RSS feed. There are directions for what specific code to use in the program.
      • Then click extract
      • Then you can give it a title, description, url, etc
      • Then put in where the title, date, etc are etc
    • If these sites are updated once a month, its too much of a hassle to make one of these (use a page monitor). But if it is updated daily and you want to monitor it, then it might be a good idea to make one!
    • Free, or $29/year for 20 hourly updates
    • My feeds

Examples

Email filtering

  1. Gmail
    • Limit around 7.2GB (4.5GB in October 2007)
    • Can use a filter
      • To forward just some emails (to different people?)
      • To apply a label to emails
    • Plus addressing
      • A powerful method that can be applied to Email alerts is using “plus addressing” service when you sign up for an Email alert (e.g. from some query), i.e. tell them that your address is dummy+moc.liamg|reifitnedIyreuQemos#moc.liamg|reifitnedIyreuQemos instead of the normal address moc.liamg|ymmud#moc.liamg|ymmud. Thus, if you get this address to your mail account, you can filter it by what comes after the plus! This is a extremely helpful since it makes it easier to filter emails.
      • Description for GMail
      • Use a different address for each email alert
      • Helps you filter
      • Helps you track who is selling your email address
  2. Defining a filter
    • Keep definition to a minimum, as simple as possible
    • Test, test, test

Tools you now have at your disposal

  • Method to follow to find site-based email alerts
  • Tools to create search-based email alerts
  • Tools to monitor Web pages for any changes to their contents
  • Tools to apply keyword-based filters to RSS feeds
  • Tools to convert tabular Web page content to an RSS feed

Your term project

Email alerts and your term project

You should do the following for your project wiki:

  1. You should figure out some way that you are going to document the email alerts that you use in your email account to route your incoming alerts. Maybe print the alert page to a PDF file and link it to your wiki? Maybe take a screenshot of your email inbox and highlight the email alerts?
  2. In either case, you are going to want to have a section in your wiki called "Email alerts".
  3. On this page you should describe each of the email alerts that you used: the page from which you subscribed to it, why it is useful, and if there are any keywords (or such) that you used to generate it.

All of the above also applies to your page monitors, any feeds you create using FeedYes/Feed43/Dapper, and any feeds you filter using FeedRinse or Yahoo Pipes.

Possible blog topics

You do not have to write a blog. These are suggested blog topics if you were to write one. There are lots of possibilities in this class.

  • Describe different ways that you found these tools useful (or not useful).
  • Describe how you used Yahoo Pipes, possibly differently than how we have described them here.

Hints about possible test questions

You're definitely going to be held responsible for the following topics:

  1. What WatchThatPage (as an example of a page monitor) can do
  2. What Dapper can do
  3. What Feed43 can do and how its search patterns work
  4. What Yahoo Pipes can do and how feeds can be manipulated (for example, Fetch Feeds, Union, Filter, Sort)
  5. Under what circumstances would you use each one of these tools (as opposed to another)
page_revision: 10, last_edited: 1258988750|%e %b %Y, %H:%M %Z (%O ago)
EditTags History Files Print Site tools+ Options
Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License