Monday, 5 March 2018

Open Graph (og:), Twitter Card and more meta data being collected by Integrity Pro and Scrutiny

Yesterday I wrote about the development of Integrity Pro.

One of the enhancements to come in Scrutiny v8 is a load more data about your links and pages,  part of that is the collection of most of the meta data from your pages.

These are some screenshots showing how Integrity Pro displays meta data, as an additional table under SEO, and within the 'page inspector' which pops up when you click one of the pages in the 'links by page' view.

The reporting of og: and twitter card information has been on the Scrutiny enhancement list for a while, and will be in v8 as you see it here in these shots.

Integrity Pro, which will also contain this functionality, will be out for public beta very shortly.

Sunday, 4 March 2018

Big news - Integrity Pro

Recently I wrote about the new releases of Integrity and Integrity Plus and how, despite being +2 major version numbers, they're unlikely to get many people this excited.

Something that I hope will light more fires is Integrity Pro! It's not intended as a replacement for Scrutiny, but it will bring the SEO and spellcheck functionality into Integrity and be priced midway between Integrity Plus and Scrutiny.

Thus making it a natural and affordable step for Integrity Plus users. There will be a fair upgrade path for Integrity Plus users, and free for any licensed Scrutiny user who wants the more classic interface and doesn't need the full weight of Scrutiny.

Integrity and Integrity Plus v8 are available now. Integrity Pro is nearly ready. Here are some screenshots).


Sunday, 18 February 2018

Integrity v8 engine under development

The beta of link checker Integrity v8 is imminent, with Integrity Plus following closely behind and then website scrutineer Scrutiny.

[update 12 Mar 2018 - version 8 is now the full release in Integrity and Integrity Plus]

Obviously we've been looking forward to version 8 so that we can say 'V8 engine'. The biggest changes in the upcoming versions of Integrity and Scrutiny are in the engine rather than interface, so there's not a huge amount to see on the surface but there is a lot to say about it.

Changes to the way that the data is structured in memory

For 11 years, the way that the data has been structured in Integrity and its siblings has meant that for certain views, tables had to be built following the scan. And rebuilt when certain actions were taken (rechecking, marking as fixed). Not a big deal with small sites, but heavy on time and resources for very large sites. It also meant that certain information wasn't readily available (number of internal backlinks, for example) and had to be looked up or calculated either on the fly or as a background task after the scan. And I have to admit to some cases of instability, unresponsiveness or unexpected side effects caused over the years by that stuff going on in the background.

So we've invested a lot of time in something that isn't a killer new feature, but will make things run more smoothly, efficiently and reliably. Initial informal tests show improvements in speed and efficiency (memory use).

More information about your links

There have been a number of requests to add more information about the properties of a link , eg hreflang and more of the 'rel' attributes (it has been displaying the 'nofollow' status but not much else). HTML5 allows loads of allowable values in the 'rel' and if the one you want doesn't have its own yes/no column in the new apps then you can view the entire rel attribute. You can switch these columns on in any of the link tables as well as the one in the link inspector.

The information displayed in Scrutiny's SEO table will include more data about the pages, notably og: meta data which is important to a lot of users.

More information about redirects

Redirect information wasn't stored permanently, only the start and end of a redirect chain (the majority are a single step. But if there is more than one redirect, you'll want to know all the in-between details)

Better control over volume of requests

It's becoming increasingly necessary to apply some limits on the rate of the requests that Integrity / Scrutiny make.

Your server may keep responding no matter how hard you hit it. That's great; turn up those threads and watch Integrity / Scrutiny go through your site like a dose of salts.

But I'm seeing more cases of servers objecting to rapid requests from the same source. And the reason isn't always clear. They may return 429 (which means 'too many requests') or some other less obvious error response. Or they may just stop responding, perhaps a defensive measure.

If that's your site, you'll no longer have to turn down the threads and use trial and error with the delay field. You'll be able to simply enter a maximum number of requests per minute, while still using multiple threads for efficiency.

Generally a little more helpful

For a while, Integrity has automatically been re-trying certain bad links, in case the reason for the first unsuccessful attempt was temporary. v8 will built on this. For example, a 'too many http requests' is often caused when a website won't work without the user having cookies enabled (this is true, I'll be happy to discuss further with examples) and in these cases, the link will be re-tried with cookies enabled and this will usually then check out ok. In the case of 429 (too many requests) they'll be re-tried after a pause (if a 'Retry-After' header is sent by the server, the recommended delay will be made before retrying). The scan will be paused if too many are received and advice given to the user. On continue, the 429s will automatically be re-tried. Once again, enhancements that will be invisible to the user.

Why wasn't I informed about Integrity version 7?

Because it doesn't exist; Integrity will skip a version. v7 of Scrutiny was mostly about the interface. Integrity continued to use the classic interface and so remained v6. Integrity, Integrity Plus and Scrutiny (and a new app, Integrity Pro, you heard it here first) will use the v8 engine when it's ready and so for consistency all apps will be called v8.

Charge for an upgrade?  Price increase?

So in summary, as far as Scrutiny v8 and Integrity v8 are concerned, the changes won't be immediately visible but they'll do what they do more efficiently and with more information available.   They'll provide more information more efficiently in a smaller memory footprint.

There are no immediate plans for price increases, or for upgrade fees. Prices are of course always under review, but any future increases will be for new users, upgrading to 8 from Scrutiny 7 or Integrity 6 will be free.

Download the beta

Version 8 (beta) is now available here. Beta means that it's available for testing and there may be bugs, so if you experience any problems, you should report them using the usual support form.

Sunday, 4 February 2018

iPhone Battery & Power Repair (UK)

I didn't find this as easy to find as I'd hoped, so I'm posting the link here.

It applies to iPhone 6 and later. The £25 offer covers the battery replacement *if they test it and decide that the battery needs replacing*. (£79 for 'other eligible models' but no word on which models.) It may be free if you're under warranty or AppleCare+. You can send it to them or take it into an Apple store.

Read more and start a request:

Thursday, 1 February 2018

MP Matt Hancock releases app called Matt Hancock MP

This story is the best laugh I've had for weeks.
I worked for local government for a while and was told "We need to be on Facebook / Twitter" by people who didn't know what Facebook and Twitter were.

I'm sure the same naivety came into play here. Just because you want to engage with people doesn't mean that they are desperate to engage with you. At least not in the way you want.

Of course users were going to find creative ways to abuse the thing. Of course there were going to be bugs and unforeseen / poorly tested trains of events causing awkward error messages.

Read on:

Wednesday, 31 January 2018

May the force not be with you (rumoured dropping of 3D Touch / force touch)

I don't like rumours but this one seems to have some credibility. It has split opinions, with some saying that they're not surprised; it's something you have to learn but can be useful when you have. Others say that it's a make-or-break feature for them. Others (I guess the majority) being unaware of how to use it properly or that it even exists.

I'm not surprised that it hasn't caught on. A long or harder press isn't analogous to 'give me a different action'. If you watch my Mum use her iPad, you'll see that it's analogous to 'shouting' the same thing that you just asked for because you didn't get a result the first time, or didn't get the result you expected. If my Mum received a popup menu or some other action as a result of trying again but harder, she'd find it confusing.

I never press to click my laptop trackpads, I have 'tap to click' switched on and hold the ctrl key if I want a context menu.

I think the 3D touch feature on the handheld devices is an unintuitive move that Micro$oft would be proud of. If Apple are dropping it, it'll be for a good reason. Maybe, like the also-rumoured dropping of OLED screens, it's a cost-benefit decision. (ie more cost but with few users feeling the benefit.)

Further reading

Sunday, 28 January 2018

Wot no toilet paper emoticon? (Emoji as a universal language)

Without going into the details of why I wanted a roll of TP emoji, I was amazed to discover that there isn't one - yet.

Back in the day I used to add a ;-) to avoid misunderstandings in emails and texts. But I'm in the "really???!" camp when it comes to adding a string of pictures at the end of a text or social media post.

When Craig gave that infamous demo of the iPhone X ("let's go to backup") He said something along the lines of "Here we are with the most advanced technology in a phone and you want me to make a poop sing" (I can't find the clip and get the exact quote). The technology has well and truly overtaken our real needs.

But then while looking at the page about the forthcoming toilet roll emoji it occurred to me that we're making a great job of assigning images to objects with a short code for each. We're building a dictionary of bytecodes for pictorial representations of objects.

That's incredibly efficient and has potential beyond embellishing your texts. How soon before we start adding more abstract concepts like 'week' and 'meeting', adjectives and conjunctions etc  and end up with a universal language which can be stored incredibly efficiently?

Friday, 19 January 2018

Limiting Integrity / Scrutiny's scan to a particular section of the site or directory

This is a frequently-asked question and a subject that's been on the workbench this week.

Integrity / Integrity Plus and Scrutiny allow you to set up rules which limit the scan, or prevent checking of urls matching a pattern:

The rules dialog* will shortly look as above; the dialog will appear as a sheet attached to its main window, it'll allow 'urls which contain' / 'urls which don't contain' and the 'only follow' option is gone from the first menu, leaving 'ignore', 'do not check', and 'do not follow'.

('only follow' was confusing because if you have two such rules, then it doesn't make logical sense. 'do not follow urls that don't contain' does the same job and makes sense if you have more than one.)

It's important to say that it shouldn't be necessary to use a rule such as 'do not follow urls that don't contain /mac/scrutiny'  if you are starting at  because Integrity and Scrutiny should only follow urls at or below the 'directory' that you start at.

It's also important to note that 'not following' isn't the same as 'ignoring'. If a link appears on a page then you will see it in the link check results regardless of whether it matches your 'follow' pattern. If you only want to see urls which follow a certain pattern, use an 'ignore' rule instead.

(That last statement applies to the link check results. The SEO table in Scrutiny and Sitemap table in Integrity Plus and Scrutiny should only show pages which meet your rules and appear at or below the directory you start in.)

Important note - if you're 'ignoring urls not containing'  then you probably won't be able to find broken images or linked files (unless their urls happen to match your rule's pattern). So if you have the 'images and linked files' option checked then you'll need to use a 'follow' rule rather than 'ignore'.

Protip - this has been possible for a while but not very well documented: you can use certain special characters. An asterisk to mean 'any number of any character' and a dollar sign at the end to mean 'appears at the end'. For example,  "don't check urls that contain .dmg$" will only match urls where .dmg appears at the end of the url. And*/mac/scrutiny will match  and

Regex is not supported (yet) in these rules (though it is available in Scrutiny's site search). It's not necessary to type whole urls or to use the asterisk at the start or end of your term. By default it's a partial match, so "urls not containing /fr/mac/scrutiny" will work.

The new "urls that don't contain" functionality is in Scrutiny from v7.6.8  and should shortly be available in Integrity and Integrity plus from v6.11.14

*Years ago, these rules were called 'Blacklist / Whitelist Rules'. I'm mentioning that in case anyone searches for those terms.

Monday, 8 January 2018

Division by zero

I've just watched this video which was embedded in this Quora answer about division by zero. The video is so entertaining I had to share it. A guy is asked to perform a division by zero on his mechanical calculator. Is it dangerous? Does the machine catch fire?
I wonder whether it occurred to Charles Babbage to try this with his difference engine.

On today's computers I'm surprised that this isn't dealt with at a high or low level. An attempt to divide by zero can crash an app and it's surprisingly easy to do. Should it be necessary for a developer to have their wits about them every time they perform a division using a divisor that is a variable? And have to write an extra line to check that that variable is not set to zero?

It would perhaps be more elegant if the processor returned the largest possible number (INT_MAX or FLT_MAX etc depending on the type of the divisor). Yes, that's the wrong answer, but we're working with approximations all the time - for example,

float f = 1.0;
if (f == 1){
 // unlikely to be executed because f as a float isn't exactly 1 - it has a 'precision',
 // in terms of its bits it'll probably be 0.999999999999 or something

There are some very interesting philosophical points in Dave Williamson's answer. Such as the fact that infinity isn't a number. Zero is more generally considered a number, but that is questionable.

All of this inspired me to draw this:
Science Museum, year 2422. Yes, this is Babbage's original difference engine. At the unveiling, he mistakenly gave it a calculation involving a division by zero. Since he hadn't thought to put any kind of cancel button, it's been chugging away pointlessly ever since

Saturday, 6 January 2018

Lifestyle for LIFX and LIFXstyle - what's going on?

This post has the relevant keywords in the title for the benefit of anyone who uses Google to find clarification.

For some years now (since Oct 2014) my app for controlling LIFX bulbs has been called LIFXstyle (pronounced Lifestyle by me but no-one else would know that and is free to say it how they like). Incidentally it's based on Hue-topia (for Philips Hue bulbs) which goes back to 2013.

I started the year with some optimism, as you do, determined to spend the time and jump through the hoops and publish more of my apps on the Mac App Store. This 'can-do' attitude was quickly knocked into touch after two rejections of this app (technically one 'more information' and one rejection). One of the reasons for rejection was the name.

It's frustrating to be faced with an imperative to change the name of something so well-established (and well-selling). So I'm not changing the name of the version sold outside of the Apple Store, unless LIFX themselves complain, which they haven't to date.

"XXXX for LIFX" was the format suggested by Apple in the rejection. The rejection was under their 'copycat' policy, I guess it was too similar to "LIFX" which is the name of LIFX's own app. Anyway, of course I went with their suggestion, keeping the new name as similar as I possibly could to the original name of my app.

That's the reason why the version available for web download is called LIFXstyle and the version on the App Store is called Lifestyle for LIFX

Friday, 5 January 2018

2018 begins with a meltdown

We have started the year with the kind of story that TV and radio love. "Nearly all computers worldwide - and many other devices - have been exposed to security flaws which leave them vulnerable to attacks by hackers" (direct quote from BBC News).

Tech correspondents were in their element, "The CPU is the 'brain' of the computer, if you like, and if someone can read everything in your brain, then they can read all of your sensitive information.." (I paraphrase only slightly).

The culprits are fiendishly-named Meltdown and Spectre.

Intel's response was the amusing classic, "It's not a bug it's a feature". The detail of it sounds much like a small boy caught red-handed, "I didn't do anything. Anyway, the others did it too."

In a nutshell, meltdown is easy to fix and has been. If your High Sierra system is up to date then you should be ok. Spectre is hard to patch and hasn't been so far but it's hard to exploit. (see Further Reading for more details.)

Install updates when available. And don't have nightmares.

Further reading:
Opinion on Intel's response:
Thorough and understandable rundown of the problems:
The BBC (who should know better) sensationalising and enjoying the chance to have a dig at Apple users:

Wednesday, 3 January 2018

New Year - Thoughts on the Mac App Store, Integrity Plus available there once again

If you've asked me over the last few years why I don't sell my apps via the Mac App Store, I'll have answered that I hate the store. One day I threw a tantrum over a problem (a catch-22  involving version numbers following a few rejections with one of my apps) and I petulantly took all of my apps off the store.

A decision made with the heart rather than the head, which is in character. But I'd been frustrated with the store since day one. Back then Apple took 2-3 weeks to review an app... Fast fixes to problems weren't possible.

Sandboxing was introduced and then made compulsory for apps listed on the Store. This is a security measure which gives you hoops to jump through and makes certain features impossible.

The iTunesConnect site (used for upload / submission) was not very user-friendly early on (let's not get into why the default music player was involved with installing apps on your computer). Apps get rejected. All of this meant that getting an app listed on the store was time-consuming and frustrating. Then Apple took a whopping 30% of your revenue.

It didn't (and still doesn't) allow for trial periods, which has always been very important for me. Developers get around this by making the app free and offering a 'pro' version (which is something I already have with Integrity / Integrity Plus) or making the app free and offering in-app purchases or subscriptions to a service. Or using advertising. None of which I really like in other apps and won't get involved with myself (another emotional rather than business decision).

I've been waiting for the day that the only way to install an app is via the Store, or by 'jailbreaking' your Mac, as with iOS. That day hasn't come (yet) but Gatekeeper has steered people away from web-downloading apps - even a code-signed app requires a 'lower' security setting than installing apps from the app store.

With all of this said, I do find things much improved.

New year - new start. I have let my logical head overrule my feelings and made the business decision to list paid apps on the store once more. The reason is simple, I want as many people to discover and use my apps as possible. In return for the frustrations and giving Apple a big cut, you get more users.

There are many things to conform with  (screenshot sizes, compulsory fields, all of which is pretty reasonable). Uploading and submitting wasn't so painful. Sandboxing still applies, as does the huge cut Apple take. I was amazed to find Integrity Plus approved within hours of finally managing to submit it.
Future plans - I don't think that Scrutiny will appear on the Store again, it has some important features which aren't compatible with Sandboxing, such as the scheduling. But I am drawing plans for an 'Integrity Pro' which adds the SEO stuff that only Scrutiny has at the moment.

Monday, 1 January 2018

Drag and drop colours from and to LIFX and Hue bulbs

If you're an existing user of LIFXstyle or Hue-topia you may think "that's not new". Versions 1 and 2 did allow you to drag colours around but with version 3 and much new functionality, the drag and drop has been missing.
The drag and drop does now carry the warmth / colour temperature value. (And of course brightness and saturation.) So if you've found a nice setting for one lamp, whether that's white with a little warmth or a full-on colour, you can drag the colour to other lamps / groups / locations.

This will be in Hue-topia 3.0.9 and is already released in LIFXstyle 3.1.1