Faster, higher, stronger

This is Scrutiny v4 checking a very large website. (Integrity 4 and Scrutiny 4 are now both available as release candidates)

The major change is to the engine. You can't see the speed that Integrity is tearing through this site but it's collected 33,000 links and checked 16,000 of them in about 15 mins. And hardly stressing the computer!

I'm dead chuffed with the improvement and also with Scrutiny's new application icon which reflects the fact that it can handle larger sites and process the data more quickly.

Note that in the screenshot Integrity is switched to the settings screen. Both apps will only keep updating the tables of results if they're visible. Constant updating is a big overhead, so switching back to settings screen makes a significant difference.

Making sure your site is fast and why it matters

I've just read this post at CopyBlogger which makes a compelling case for making sure that your web pages load as quickly as possible.

Load speed is a signal used by Google. But just as importantly, very small delays in load speed loses visitors.

Reactivity is free when downloaded from the web and tests your page, breaking it down and showing all the files it finds with load times for each (using a single thread for a more accurate measurement of each).

You'll easily spot bottlenecks - often third-party code or plugins can cause such delays and be difficult to track down.

http://peacockmedia.co.uk/reactivity/

Reactivity is a standalone component of Scrutiny which is a suite of website analysis tools.

http://peacockmedia.co.uk/scrutiny/

BASIC programming and VIC-20 emulation

I enjoy a maths problem. A friend sends me one every Friday.

The type of question determines the best way to go about solving it. I like it best when I can write out the problem as an equation and work out the answer.

Some problems can only be solved (as far as I can see) by 'testing' different numbers. This was the case today and I managed to get the correct answer with a spreadsheet. But with more than one variable, this was a bit of a tedious process.

This is where BASIC programming comes into its own. As a teenager I would have tapped out a few lines of basic into my Vic-20 and had an answer very quickly.

I've tried to solve such problems more recently using C and my usual development environment with various degrees of success.  Those used to writing CLI tools may argue that it's very easy to tap out a few lines of C, compile and run the app and get an output. But I've run into various problems, and no question about it - that's not as easy as this:

Today's problem was about sharing a box of chocolates (we don't know the total number) and one boy ends up with the same number of chocs as the percentage that his brother has. We need to know the smallest numbers for which this can be true. You can see here the two possible answers for numbers up to 100 chocs total. (20 chocs and 5 chocs (20%) out of a box of 25 and 50 chocs each (50%) from a box of 100)

I haven't written a BASIC prog like this for about 30 years, but it all flowed from the fingers very naturally. With a few more minutes of messing about I could have refined it and have a much smaller and more efficient programme.

Two points. First it's awesome to be working in such a small space. You can see that there are 3583 bytes to work in (yes, bytes. That's about 3.5k). The computer had a bit more on board but some is used by the screen. I bought an 8k ram pack for my Vic which seemed a massive amount of space at the time. (it brings to mind this challenge which involves writing a game in 140 characters!)

Second, this kind of programme could be written by a classroom of newbies in the first lesson. I'm well behind the Raspberry Pi project, but I'm not convinced it will inspire new coders like our 8-bit machines did. Switch it on with very little knowledge and within seconds you've filled the screen with your name. With a little more knowledge you'd made a small animation and after that made a simple game. My suggestion is a basic interpreter in the Applications folder of OSX and windows (and Rasp Pi). (As I have now!)

Internationalisation! ( -zation?)

Like almost all of my software, I wrote Organise to fill my own needs and still use it heavily myself.

Occasionally someone has contacted me about problems with calculations if their system preferences are set to format numbers differently from here in the UK (for example, if they're in a country using a comma as a decimal separator and dot '.' as a thousand separator).

But I've never gone through the app and made sure that Organise works perfectly regardless of user number formats. The same thing applies to the sales tax rate (for example, Canada apparently has three different taxes, which Organise hasn't been able to handle).

I had a wake-up call when Organise was put on offer on a popular download site recently. Why people buy software without taking advantage of a free trial I don't know, but they apparently do, and it's very awkward when they find that it doesn't work for them.

It has been a tough task. The code behind Organise is vast now and there are many places where it makes calculations and needs to take account of the user's choice of number format. But I'm just about there.

And to complete a truly international version (version 6!), I've also tried to add the flexibility that's needed for folks to calculate their sales tax wherever they are. Defaults are in line with the UK rules, but as you can see here it's now as customisable as I think it needs to be.

I hope to have a release candidate ready for download in the new year.

disappointment with iTunes 11

I'd like to give some link love to Joe of eMac Consulting.

His procedure for 'upgrading' iTunes 11 back to 10 worked perfectly for me

I've been open about the fact that I enjoy using Snow Leopard for my day-to-day work. I do think it is the height of Apple's powers and I don't like the general direction since Lion.

After reading about iTunes 11 I couldn't resist getting hold of the shiny new interface.

I think the lack of coverflow was the last straw, but before I realised that I'd started to feel disappointed with the interface.

I'm prepared to accept that this may be a reluctance to adopt change, but for the record here are a few things that baffled me:

• Your music list is data - why not be able to change the way you view that data (list, album list, grid or coverflow) easily with buttons at the top of the window, consistent with Finder?
• The box at the top was a good analogy for an LCD display. OK, it no longer has the LCD look (what is it meant to look like now?) but it just doesn't seem right to have controls in there (eg the shuffle button)
• The play, next and previous buttons no longer look like physical buttons - there's nothing to tell you that they're 'pressable'. I can't see any reason for this change other than 'for the sake of it' but whatever the reason, why still have the volume slider looking 3D physical thing?

I've fallen back in love with iT10 and that's a lesson learned about trying these new things out on my Mountain Lion machine (or rather partition) before making 'difficult-to-reverse' changes here on my lovely Snow Leopard.

sniffing out difficult-to-find broken links

I thought it might be good to document one of the more obscure features of Scrutiny.

The Microsoft Content Management System (or at least the system that I've had the misfortune to be on the editorial end of) had an odd way of dealing with broken links.

If it detected a bad link when generating the page, it would replace the expected 'href' within the link with 'zref'. <a zref = "foo.com">foo.com</a>

This would prevent the user from experiencing a broken page, but they had a link that they could click with nothing happening, and made it impossible to track down broken links.

I don't know whether this thinking is a part of MCMS or part of the system that our providers had built on top of it (and may still be providing to others).

But Scrutiny will find and flag such links.

Do you use a system that makes broken links hard to find?

Server returning 400 for url with no referer

I've had an interesting support problem this morning and thought that it might be useful to log the answer here.

The problem was Scrutiny not being able to get past the starting url - reporting '400 bad request'. But the same url would return the expected page in a browser.

It seems that this particular server doesn't like not being sent a 'referer' field. Scrutiny does send a referer for all other pages it crawls, filled in with the url of the page that the link appears on. But by definition there is no referer for the starting url and at present it doesn't send one.

Going to Advanced settings and entering 'referer' as the name of the first custom header field, with any valid url (including 'http://') as the value then the crawl worked.

Sending an empty string or a space for the value doesn't seem to work, so I'm not sure what the browsers do (If anyone knows the answer to this I'd be grateful)