Scoring Startupware
It should be possible to rate individual products as startupware. Not just good or evil–that’s not it. What’s needed is a measure of how invasive they are, and how hard to remove.
Remember that this stuff isn’t all spyware; it includes antivirus software, overly-ambitious print drivers, and it’s not all evil, although most of it is bad, all of it need managing.
What’s that? Antivirus is never bad? Wrong. If in doubt, install two of them on a clean system, and try to do some work. Be sure to refresh your memory on safe-mode cleanups first, as most combos of this type will turn a computer into a vibrating doorstop. Like all startupware, it’s a management task.
To help consumers decide what products may be allowed on their systems, a scoring method is helpful. Scores skip technobabble–that’s good. They also can cause a blanket reaction of “take out all of it, don’t bother me.” That’s usually OK, but I don’t want the phone call when that breaks the antivirus software.
The basis for using startupware as a management tool for software products and their accumulations of autoplays is that no judgement calls are allowed. Again, here is the definition:
stärt’-up-wãre, noun, any software that configures portions of itself to automatically start with the operating system of a computer, or to start with other previously-installed software.
Now, there are different ways to autostart, and it helps to know if a product cleans up its own mess on removal, so let’s find a way to score a program for startupware.
First, we need to keep track of how many programs are set to run on system startup, and if all of them are removed on uninstallation. If one program is installed, but results in two add/remove program entries, that’s backpackware, which is common in adware and spyware products, as well as simpler trojan horse programs.
Here’s a preliminary formula for Startupware scoring (version 0.1) .
Orphaned programs: 1000 x number of programs installed to autorun but not uninstalled by removing the product that was chosen to be installed. Note that one install program that results in two or more product installations will always result in a high score for deceptive behavior. Exception: One install that offers to run an additional OPTIONAL install program is counted as more than one install program, so that, for example, a camera driver install that offers to install a graphics program is counted and scored as two installations.
Orphaned settings and silent programs: 100 x number of settings changes made, but not uninstalled, and number of programs that run when product isn’t doing work for the user, such as displaying information or being on standby to do or prevent something.
Autorun count: 10 x number of programs installed to autorun.
Settings count: 1x Number of settings changes made.
comma, “version†and number of program, or “tested†and 6-digit date downloaded (yymmdd) if no version number is used.
So for some program categories, it’s impossible to have score of 0, which would be totally non-invasive and non-autoplaying; a screensaver would have a score of 11, minimum, and so would most system tray utilities, because it takes both a program and a setting change to have an autoplaying program. And some actions aren’t counted. There is no count of icons added to the desktop, the quicklaunch area, or to the menus. There is no count of file extensions modified to point to the new program.
Here are some examples: a toolbar program, with no version number, with one program running in the background while the toolbar was not on screen (100 points). It made 12 changes to system settings, and failed to uninstall 1 of them (12 + 100 points). Total 212 points.
Score: 212, tested 050825.
Example 2: a utility program, installs two programs that don’t autoplay and don’t run in the background, changes no settings, leaves no settings or programs behind, version number 2.1.
Score: 0, version 2.1.
Example 3: an application program, version 12.0. Installs 17 programs, 3 autoplaying. Uninstalls all of them. Makes 32 settings changes, removes 12 of them. (Typical big-product sloppyness, in short.) That’s 30 points for autoplays, 32 for settings, 2000 for orphaned settings, and no orphaned programs.
Score: 2062, version 12.0.
Note that spyware won’t always get the highest scores. Startupware is about invasive software that drags down system performance, and not about subtlety or theft.
Example 4: a screensaver, no version number, downloaded Sept 10, 2005. Installs one autoplay program, clean uninstall, one setting change that runs the screensaver.
Score: 11, tested 050910.
Example 5: printer driver, version 18.544. Installed 3 autoplays, left one behind on uninstall, 71 settings changes, 26 left behind. That’s 1000 + 2600 + 30 + 71
Score: 3701, version 18.544.
Example 6: anti-spyware program, bundled with toolbar with no option to install only one, installed with one program but resulting in two entries in add/remove list. That’s backpack startupware, and if no permission was asked first, it’s stealth startupware. Determining Stealth or Backpack isn’t needed–it depends on disclosures and agreements, and doesn’t affect product behavior, and so doesn’t affect scoring. There are 3 autoplays, and the uninstall that matches the downloaded product removes one of them. Settings changes: 15, 10 left behind. That’s 2000 points for orphaned programs, 1000 points for orphaned settings, 30 points for autoplays, and 15 points for settings.
Score: 3045, tested 050901.
Essentially, what we’re trying to achieve is a high score for programs that fail to uninstall themselves completely, or that massively invade the system. In other words, don’t install a program with a score above 50.
Comments? Additions? Modifications?
2 Comments »
RSS feed for comments on this post. TrackBack URI
Leave a comment
Line and paragraph breaks automatic, e-mail address never displayed, HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>
June 16th, 2006 @ 12:32 pm
I like the approach you’re taking, but it will trigger ill feelings among companies that correctly uninstall, properly identify everything being installed, and yet get a moderately high score because of the auto-run application count and settings count used by their product.
I don’t see the need to count any settings changes if they are uninstalled correctly. If the application chooses to use many of what you have called “settings” (but haven’t defined, mind you), how is that bad for the speed of the machine? The registry might be larger, so everything using the registry could be affected slightly by the bloat, but that’s not an ongoing effect once things have initialized.
Note that there are likely to be some settings that cannot be undone if other applications overwrite them since the application being uninstalled changed them. You can’t count those as not being uninstalled. A prime example is extension mappings. You should clarify that an application is only responsible to undo what it has done to settings.
The scoring should account for memory and CPU utilization. An application running several small auto-run programs that are efficient consumers of memory and CPU can have a higher score than an application with just one auto-run program that’s a resource pig. The former would be more appropriate to run, if you really needed to choose one or the other, so the scores should reflect that.
June 16th, 2006 @ 12:58 pm
Stew–
For scoring purposes, you’re absolutely right that system changes cleaned up by uninstall (or not) have to be counted with NO other programs added or removed during the time that the program being tested is on the system, or there won’t be either fair or consistent results.
I’ll also agree that one high-CPU-use startup entry is far worse than 4 entries in the ‘extra button’ category. (And so on. Mileage will vary.) But some kind of scoring is needed, and CPU utilization can’t be easily measured for an installation package; the process of monitoring throws off the accuracy of the measurement, and the effects of any startupware on CPU usage will vary with the speed of the test box. (The Heisenberg principle comes to computers…)
Something simpler is needed, and that does mean something arbitrary; can’t be helped. Suggestions?