Showing posts with label website monitor. Show all posts
Showing posts with label website monitor. Show all posts

Thursday, 19 September 2019

New feature for Website Watchman and spin-off website archiving app

I wouldn't be without Website Watchman scanning the PeacockMedia website on schedule every day. As with Time Machine, I can dial up any day in the past year and browse the site as it appeared on that day. And export all files associated with any page if I need to.

It also optionally alerts you to changes to your own or anyone else's web page(s).

It took a lot of development time last year. I have (literally) a couple of reports of websites that it has trouble with but on the whole it works well and I think the tricks it does are useful.

Since it went on sale early this year, it has sold. But not in the numbers I'd hoped. Maybe it's just one of those apps that people don't know that they need until they use it.

Here's the interesting part. Of the support requests I've had, more have been on one question than any other. Frustrated emails asking how to make it export the entire site that they've just scanned.   What those people are after is an app which does a 'one-shot' scan of a website and then saves all the files locally. It's a reasonable question because WW's web page talks about archiving a website.

My stock answer is that WW is designed to do two specific things, neither of which is running a single scan and exporting the entire thing.  Like Time Machine, It does hold an archive, which it builds over time, and that's designed to be browsed within WW and any files that you need from a given date are recoverable.

There is of course quite a well-known and long-standing app which does suck and save an entire site. I've had conversations with my users about why they're not using that.

So knowing about those requirements, and owning software which already nearly does that task, the time seems right to go ahead.

I've mused for a while about whether to make an enhancement to Website Watchman so that it can export the files for an entire site (remember that it is designed to make multiple scans of the same website and build an archive with a time dimension, so this option must involve selection of a date if more than one scan has been made) or whether what people are after is a very simple 'one-trick' app which allows you to type a starting URL, press Go, choose a save location and job done.

I've decided to do both. The new simple app will obviously be much cheaper and do one thing. Website Watchman will have the new full-site export added to its feature list. I'm using WebArch for the new app as a working title which may stick.

So where are these new things?

It's been more than a week since writing the news above. Even though we already had toes in this water with the archive functionality (such as it is) in Integrity Plus and Pro and Scrutiny, it turns out that it's no easy thing to make a local copy of a website browsable. In some cases it's easy and the site just works. But each site we tested was like opening a new Pandora's box, with new things that we hadn't taken into account. It's been an intense time but I'm really happy with the stage it's at now. I'd like to hear from users in cases where the local copy isn't browsable or problems are seen.

The new functionality is in Website Watchman from version 2.5.0. There's a button above the archive browser, and a menu item with keyboard shortcut under the File menu.

The new simple app is WebArch and is here for free download.

Monday, 4 March 2019

Website archiving utility, version 2

Watchman for Mac is a utility that can scan a single page or a whole site on schedule, it'll archive the page(s) and alert the user to any changes in the code or visible text.

As it builds its archive it's possible to browse the historical versions of the pages, as they stood on particular dates. It displays a 'living' version of the historical pages, with javascript and stylesheets also archived.

We've just made a version 2 beta available. It features a 'non-UI' or headless mode, which means that it can remain in the background and not interrupt the user when a scheduled scan starts. Its windows can still be accessed from a status bar menu.

https://peacockmedia.software/mac/watchman/

Version 1 is still available. It's free and will remain free. The new beta is also available and free at present, but version 2 will eventually require a licence.

Wednesday, 9 January 2019

New website monitor / archive utility for Mac arrives at full stable release and is still free

Watchman is an easy-to-use website monitoring / archiving utility.

You can use it to watch a single page, a part of a website or a whole website. It can run on schedule (hourly, daily, weekly, monthly) and alert you to the changes you're interested in, which could be visible text, source, resources appearing on the page, its status, or you can simply leave it to archive all changes to all files. You can set up multiple website configurations, each with their own schedule. It uses the system's launchd, meaning that Watchman doesn't have to be left running, it'll just start as needed.

Watchman uses the same fast, efficient crawling engine as Scrutiny and Integrity, which has been developed over 12 years and offers a huge amount of configuration and tuning. This is coupled with a new web archive format.



Its web archive format can store changes like a Time Machine backup. You can view any page as it appeared on a certain date. When you do so, you're viewing a 'living' version of the page, with its css and javascript running as it was at the time, not a simple screenshot. You can of course export a version of a page as an image, or as a collection of all the files under their original filenames, as they were on that date. You can switch between versions of a page to compare them.

It's a desktop app running on your own Mac, so you own your own data.

It's early days, there are many more features in the pipeline, but for now it's stable and doing invaluable work. And version 1.x is free to download and use. (The next major version may not be free, or may be 'freemium' but the current version will continue to work and remain free.)

I've been flagging the release of Watchman for a while, it's been a long time since I've been so excited about a new project and I believe it'll become a more important title for us than Scrutiny.

https://peacockmedia.software/mac/watchman