New feature for Website Watchman and spin-off website archiving app

I wouldn't be without Website Watchman scanning the PeacockMedia website on schedule every day. As with Time Machine, I can dial up any day in the past year and browse the site as it appeared on that day. And export all files associated with any page if I need to.

It also optionally alerts you to changes to your own or anyone else's web page(s).

It took a lot of development time last year. I have (literally) a couple of reports of websites that it has trouble with but on the whole it works well and I think the tricks it does are useful.

Since it went on sale early this year, it has sold. But not in the numbers I'd hoped. Maybe it's just one of those apps that people don't know that they need until they use it.

Here's the interesting part. Of the support requests I've had, more have been on one question than any other. Frustrated emails asking how to make it export the entire site that they've just scanned.   What those people are after is an app which does a 'one-shot' scan of a website and then saves all the files locally. It's a reasonable question because WW's web page talks about archiving a website.

My stock answer is that WW is designed to do two specific things, neither of which is running a single scan and exporting the entire thing.  Like Time Machine, It does hold an archive, which it builds over time, and that's designed to be browsed within WW and any files that you need from a given date are recoverable.

There is of course quite a well-known and long-standing app which does suck and save an entire site. I've had conversations with my users about why they're not using that.

So knowing about those requirements, and owning software which already nearly does that task, the time seems right to go ahead.

I've mused for a while about whether to make an enhancement to Website Watchman so that it can export the files for an entire site (remember that it is designed to make multiple scans of the same website and build an archive with a time dimension, so this option must involve selection of a date if more than one scan has been made) or whether what people are after is a very simple 'one-trick' app which allows you to type a starting URL, press Go, choose a save location and job done.

I've decided to do both. The new simple app will obviously be much cheaper and do one thing. Website Watchman will have the new full-site export added to its feature list. I'm using WebArch for the new app as a working title which may stick.

So where are these new things?

It's been more than a week since writing the news above. Even though we already had toes in this water with the archive functionality (such as it is) in Integrity Plus and Pro and Scrutiny, it turns out that it's no easy thing to make a local copy of a website browsable. In some cases it's easy and the site just works. But each site we tested was like opening a new Pandora's box, with new things that we hadn't taken into account. It's been an intense time but I'm really happy with the stage it's at now. I'd like to hear from users in cases where the local copy isn't browsable or problems are seen.

The new functionality is in Website Watchman from version 2.5.0. There's a button above the archive browser, and a menu item with keyboard shortcut under the File menu.

The new simple app is WebArch and is here for free download.