Category: Geek stuff

If it involves physics, math, computer science, electronics or has general geek cachet, this is the place for it.

Suspension of disbelief and Westworld

The suspension of disbelief has a particular peculiar character within the science fiction genre. While there is certainly sci-fi that rejects all standards of realism rooted in actual science, and which might thus be better seen as a kind of fantasy with technology, most sci-fi seeks to imagine things that could be possible in the real universe, at least if the requisite technologies and aliens show up.

When experiencing science fiction, I find myself always cataloguing two kinds of consistency and places where each breaks down. First there is consistency within the world established by the narrative. If the robots in chapter one can be easily fooled by colour photocopies of people’s faces it shouldn’t change for no reason in chapter two. In a broader sense the internal rules of the universe should be consistent. If multi-year travel times between settlements are a major part of a fictional universe, the economic and political life of the settlements should be compatible with that. The second kind of consistency is with the known rules the real universe follows. This is routinely violated by sci-fi with comic book or action hero physics, where the capabilities of technology depend on the emotional stakes and the needs of the plot, rather than serving as a template for what the characters are free to do.

I have watched the first two seasons of Westworld with both kinds of consistency in mind and have been much more frustrated by internal inconsistency than by straight-up scientific impossibility. Perhaps with the big exception of “what powers the hosts?” the show doesn’t pose many straight-up problems of practicality. Rich and determined enough people could do most of what has been depicted so far (ignoring the question of whether copying and creating conscious beings is possible as depicted). Since a lot of the show is shoot-em-up gore, perhaps the most frustrating internal inconsistency regards what it takes to actually kill the robots (called “hosts”) and specifically why damage to their physical bodies can in any circumstance damage the small protected orbs which are supposedly their brains.

It makes sense in the emotional and Western contexts that one well-placed bullet brings down anybody, but it doesn’t make sense anymore when the constraints that are supposed to make the durable, re-usable robots into suitable targets are no longer being applied, and especially when some robots just shrug off bullets now because they have been reprogrammed. We’re two seasons in and everybody is still being killed because their tougher-than-humans replaceable bodies get damaged in ways that would make a person bleed to death or otherwise no longer be able to keep vital organs functioning.

Probably the writers have an answer or will roll one out subsequently, while most fans will put it down to the rule of cool on the basis of wanting to see more human-style gunfights between the robots. To me it comes across as unsatisfying, however, and a failing or unwillingness to think through the implications of the premises which the writers have already established. They’re taking a lot of the Ghost in the Shell universe where “bodies are a dime a dozen”, but sticking with gunshot wounds as a mechanism to sometimes-permanently sometimes-temporarily kill robots. The inconsistent treatment of guns is unsatisfying in other ways too, like how apparently there was some system built into the park to keep real guns from injuring human guests (suggesting some omnipotent operating system controls everything in the park) but which one person then shuts down in only that very limited way. The way they control explosives doesn’t make sense either, with the control room approving one explosion specifically for an important guest, but robots apparently playing with real nitroglycerin in several of the park’s programmed narratives. If guests are interacting with real wagonloads of nitroglycerin, how do they not get routinely blasted to pieces? And if the park can control how badly humans versus robots are hurt by nitroglycerin explosions, why don’t we see the evidence of that kind of control in other places?

It’s basically standard in fiction that characters important to the plot are impossibly competent with their weapons, while anyone attacking them is impossibly incompetent (like the much-mocked stormtroopers in Star Wars), but this is taken to an implausible degree when a single person with an antique pistol kills whole squads of mercenaries with submachine guns before any of the mercs can notice what is happening and use a weapon.

Science fiction is meant by many authors as a means of exploring philosophical ideas, as well as the implications of technology, and allowing inconsistencies and implausibilities may be intended to serve that purpose. That’s fine as far as it goes, and it’s not for me to tell authors what plot contents are or are not appropriate in their creations. Still, to some degree the task of creative worldbuilding depends on the contents holding together with each other and when that common basis is eroded by inconsistent treatment it diminishes the plausibility and immersiveness of the entire world.

Text editing at 5K

My replacement desktop computer arrived six days early. After a lot of thought about “pro” options (the crazy expensive iMac Pro, the dated but in some ways capable Mac Pro, the mysterious new Mac Pro to come) and hackintoshes, I replaced my 2007 era 2.8 GHz Intel Core 2 Duo iMac along with an Nvidia GEForce 8800 GS with 512 MB of video memory with a new iMac. This one has a 4.2 GHz quad core i7 processor (with hyperthreading) and a Radeon Pro 580 with 8 GB of video memory.

This will be my all-purpose system, for use in writing my PhD thesis, editing my commercial photography, and even running Starcraft II and Kerbal Space Program on a system that won’t grind, puff, and crash after less than five minutes.

Moore’s Law had me hoping that an eleven year delay between computers would mean getting a 30 times performance improvement, but my research online suggests it may be more like 40% for a lot of purposes. When I have finished configuring the new system I will run some comparative benchmarks. I’ll admit that it’s pretty spiffy to launch Chrome with Quicksilver and have the window open *immediately*.

This is my fifth major computer system (not counting cell phones, Kindles, consoles, or Raspberry Pis). When I started at UBC I got a Windows PC called 3Jane which got left in Vancouver when I moved to England and may well still be sitting in a closet at my parents’. My Oxford-era computer was an iBook which I named Zaphod which was eventually sold after several repairs and upgrades to Tristan’s brother. I bought the old iMac (Seldon) with one of my first real paycheques as a government employee, and added a Macbook Pro (Holmes) when I started my PhD. Planning to stick to the fictional character naming scheme, aiming ideally for a female character, and considering all the secondary devices that already took “Motoko” and “Lyra”, I considered Olmy from the Eon universe, Lucifer from Paradise Lost, HAL, and Alia from the Dune universe before going with my older idea of “Galadriel”.

I hope it will function as well in 2029 as Seldon still does today: frustratingly slow for multitasking and incapable of gaming, but still an adequate system.

How this site broke and got back online

The world is now full of technology that needs regular software updates to fix security vulnerabilities as they are publicly reported. This includes all of your computers (including cell phones, smart devices like TVs and sensors, and network equipment like routers). It definitely applies to website content management systems (CMS) like WordPress.

That’s why when WordPress 4.9.4 was released in February, my hosting provider DreamHost implemented my ongoing instruction to automatically update the software.

How WordPress works

For those who don’t know, WordPress stores posts, comments, and all sorts of other things inside a database based on open-source software called MySQL. The other big piece of WordPress is the programming language PHP.

You can think of the MySQL database as where WordPress stores everything it knows, and PHP as the machinery that lets WordPress operate and serve up what you ask for. You might think of websites as being like newspapers: all set up and formatted before you have anything to do with them. Actually, modern websites are created dynamically as your web browser talks to the server and the software on the server makes decisions about what to send you.

For example, consider the web address:

https://www.sindark.com/page/50/

WordPress is set up to show a certain number of posts per page, and then to allow users to scroll back through older pages if they wish.

When your web browser visits https://www.sindark.com/page/50/ some pretty complicated stuff happens on the server side. It works out how many posts there should be on each page, works out what should be on the 50th page, goes into the MySQL database to find the titles and contents of posts, as well as their authors and the number of comments on them, and then it puts together an HTML page which your browser displays.

Exactly how everything looks visually in WordPress depends a lot on themes. These are collections of files that tell WordPress what typefaces to use, where to locate design elements, what to show on each page, and more.

For years this site used a premium paid theme called Thesis. Specifically, it used the latest version of Thesis 1. Sometime around 2012, Thesis 2 was released but, whereas Thesis 1 allowed non-expert website operators to set up the look they wanted with simple menus, in my opinion Thesis 2 doesn’t help all that much in designing a look from scratch and requires essentially a web designer’s capabilities to use.

So, the site was using what was arguably an antiquated theme before the WordPress 4.9.4 update was installed.

How the site broke

For someone as non-expert as me, a big piece of software like WordPress is like a space station. It’s complicated and I don’t begin to understand how most of it works, but I can see when things have gone badly wrong, like because the station modules are full of smoke or, worse, literally nothing.

WordPress themes store information in the MySQL database, such as the location and content of menus.

It’s not that Thesis 1.8.9 (the latest version of Thesis 1) is incompatible with WordPress 4.9.4. My old climate-focused site BuryCoal still uses the theme and upgraded just fine, as did my professional photography site durablepigments.com.

Computers make a lot fewer mistakes than humans, but they do happen. A file you download can have some of its contents incorrectly transmitted, and a processor can perform an operation incorrectly on data. Of course, bugs in software can produce errors too.

First, some kind of error broke the back-end system that allows a WordPress site operator to create new posts, manage comments, change how the site looks, and so on. At that stage, all the visitor-facing parts of the site still looked normal. I just couldn’t manage the site from my end as usual.

I put in some effort trying to fix the site, eventually leading to it going down completely. This highlighted the importance of not allowing my ignorance and DreamHost’s limits to permanently wreck the old database. It had problems that kept the site from working, but it was still a good copy of all my posts and all your comments.

Fixing the site

Job number one was to avoid destroying all the years worth of content on the site. Tinkering with the MySQL database, undertaken by an absolute non-expert, carried a considerable risk.

This site is hosted using the least expensive plan DreamHost provides, which is called shared hosting. The name is a little misleading, because even sites on more expensive plans “share” the computer server where they operate with other sites. Those higher-end plans, however, promise you a certain amount of resources like RAM. On shared hosting, an unknown number of sites are all sharing those resources which, among other things, makes it possible for a big jump in popularity on some totally unrelated site to slow down yours.

Shared hosting has other limitations. Crucially, in this case, DreamHost limits which tools you can use to work with your MySQL databases. Through their website they provide a tool called phpMyAdmin which theoretically lets you do things like modify the content of databases, export their contents, and import contents into a new blank database.

Unfortunately, phpMyAdmin suffers from one huge limitation that crops up commonly in shared hosting. If you ask the server to handle too much data, it gets overwhelmed and gives up. This happens to me constantly when I try to upload photos to the site (indeed, that frustration is a big reason I have been considering leaving shared hosting and/or DreamHost). For a site with as many posts and comments as this one, a lot of what I read online suggested that this could be a problem. One major alternative — copying the database using Secure Shell (SSH) isn’t allowed for shared hosting users.

At the beginning of March, I was struggling with efforts to make a copy of my MySQL database to tinker with without risk of breaking the original.

There’s actually a bigger problem, though. Think for a moment about a typewriter. It has all the letters of the alphabet, punctuation, and probably some special symbols like & and ^. With computers, there are different character encodings which similarly include letters and symbols. A basic one, ASCII from 1963, doesn’t handle much more than the typewriter. It basically includes Arabic numerals 0–9, upper and lowercase letters from the English alphabet, and standard punctuation.

But people use computers in languages other than English which include diacritical marks and characters not used in the English alphabet. People also use special punctuation marks like endashes and emdashes. Partly for these reasons, Unicode was developed in the 1980s, eventually allowing people to use all sorts of characters. WordPress, like many computer systems, now uses a UTF-8 character encoding.

To summarize: WordPress is software that helps you turn content like the text of blog posts into a website people can access. It stores that content in a MySQL database, and the content of that database is encoded using UTF-8.

This next bit is a little tricky and probably won’t have occurred to most web users. Using a system like UTF-8 can be risky in a variety of ways. For example, it contains characters from foreign alphabets which look indistinguishable from English letters but which are known to be different by computers. This could allow somebody, for instance, to register a website that looks visually like google.com but which is actually run by the person who made the site with the non-English characters.

Even when it comes to importing new content into a MySQL database UTF-8 could cause problems, so phpMyAdmin will take certain non-standard characters and replace them with what looks like gibberish on import. So, the Greek letter delta imported into phpMyAdmin becomes Δ and `smart’ quotes, which I hate because of these kinds of problems, but which the Thesis theme uses, turn into “ and â€.

So, even when I succeeded in importing my old database into a new one (to be able to fix the site without risk of breaking the original), the new version contained many thousands of errors. I didn’t want to keep adding to a site full of errors, since I realized it should eventually be possible to get a properly copied database.

More on encoding and the web:

The fix

Anyway, it turned out that the pretty basic steps I had been asking DreamHost to use all along worked fine as soon as I found a customer service representative willing to read through and implement them.

I’m not the first person who had this problem with character encoding and phpMyAdmin. Early on I found a website called Orthogonal Thought which describes the problem and some ways to fix it.

Unfortunately, the fixes are done via SSH, which DreamHost doesn’t allow with MySQL for those on shared hosting. I had to get someone on DreamHost’s side to run these commands.

And so began an agonizing process of submitting customer service ‘tickets’, as requests for help are often called in the world of information technology. In each I tried to explain what the problem was, and in each I directed the tech support person to the post on Orthogonal Thought along with a request that they make a copy of my database with characters intact.

DreamHost tech support person after tech support person then did one of three things: refused to help because they thought this problem was something I should fix (despite how the necessary tools are denied to those on shared hosting), made a copy of the database where the character encoding was still broken, or made a copy of the database that somehow didn’t work with WordPress at all. In March, “John R” gave me the “not our problem” treatment, while the efforts of other tech support personnel yielded a set of unfixed databases through April and May.

I sought help from other forums and expressed my frustration on Twitter, leading to many messages from other web hosting companies explaining how bad DreamHost shared hosting is. In many cases, the people operating Twitter accounts for other hosting companies provided me with tech support via Twitter, trying to find ways to copy the database properly myself.

After months with the site down, in desperation I started tweeting at all the people who describe themselves as DreamHost employees in their Twitter bios like @DreamHostBrett whose Twitter handle is in their newsletters, “WordPress Core Developer” Mike Schroder, and “Product Marketing Manager” Jennifer Kay. None of them responded to me, but this prompted another round of exchanges with the DreamHost tech support Twitter account @DreamHostCare.

Finally, a day ago, one of their tech support people emailed me to say they had made a good copy of the database. Indeed, they finally had.

Aware that other people have had and will have this problem, I asked for the solution they used and was told by email: “Per our manager “I made sure to include –default-character-set=latin1” and changed it to “changed latin1 to utf8″”. They had used one of the fixes from the blog post which I had been sending them all along.

There doesn’t seem to be much appetite at DreamHost for looking into and fixing problems with their customer service. That plus all the site reliability problems that have cropped up due to shared hosting over the years have me still searching for alternatives. Probably, I will test out another hosting provider with a set dedicated to my PhD research and move everything over there once I am confident it’s better.

I hope some people from DreamHost will read this and reflect on what it says about the effectiveness of their tech support. One huge problem is how every time a new ticket is created it seems to get randomly assigned to someone new who doesn’t understand the background to the problem. I have been told there is also no way to elevate the problem to the attention of a manager when it proves beyond the capabilities of the first-line tech support people. Unwillingness or inability to follow simple instructions has been the problem all along here, and I would like to hear that they have some intention of making things better.

If they want to credit me back for the nearly four months my site was down, I would be open to that too.

Open thread: can life be simulated?

Slashdot reports: “Researchers at the Technische Universitat Wein have created a simulation of a simple worm’s neural network, and have been able to replicate its natural behavior to completely mimic the worm’s natural reflexive behavior.”

When it comes to bodies, at least down to the level where quantum uncertainty becomes important, there seems to be no reason why sufficiently powerful computer hardware could not produce an excellent simulation. In the long term, that could allow things like the development and practicing of surgical techniques on simulated bodies; improved testing for safety in diverse applications; and research into animal physiology.

Perhaps brains are different; there may be something about consciousness that keeps it from being modelled by any conventional computer, regardless of its memory and processing capabilities. Still, it’s possible that consciousness can also be simulated, or re-created in a digital form, perhaps with the aid of quantum computers.

Related:

The Fourier transform

Richard Dawkins’ Unweaving the Rainbow includes a great discussion of the scientific uses of the Fourier transform. Most amusingly: “The side-to-side waving of the urine trail on the road was presumably produced by the long [elephant] penis acting as a pendulum (it would be a sine wave if the penis were a perfect, Newtonian pendulum, which it is not) interacting with the more complicated periodicity of the lumbering four-legged gait of the whole animal.” (p. 73)

This video provides an accessible visual explanation of how this mathematical tool breaks down a complicated curve into its constituent sine waves, and some of the useful applications for that transformation:

Urban mesh networking

One fascinating dimension of software-defined radio is the ability to establish mesh networks: distributed data sharing systems where each computer involved is a node which can carry traffic on behalf of others. That means that as long as you have solid radio links you can establish a network that can transmit information independently from the commercial internet, run by the kind of telecom companies that provide home internet connections. If you then connect some parts of the mesh to high-quality internet connections, you can share internet access over the mesh network.

This is all part of the plans of Toronto Mesh, a group that meets at Robarts Library and is planning to set up such a network in Toronto. NYC Mesh is much father along: with Manhattan and Brooklyn ‘supernodes’ in place which provide internet access through the mesh.

There are numerous advantages to a mesh network. It can free people of all the bad behaviour from local telcos: charging monopoly prices, slowing down traffic from some sites, engaging in surveillance themselves or supporting government surveillance, etc. It also holds the promise to create more resilient networks which are better able to cope with societal disruption. Building infrastructure of that sort will be important as climate change continues to destabilize human and natural systems.

I’m collaborating with Toronto Mesh to propose a hardware and software development partnership with the Campus Co-Operative Residence. They have a large number of houses within 1.5km of each other, share many of the values of Toronto Mesh, and would likely value the ability to control and enhance their internet access in the ways mesh networking would allow. The proposal is circulating for comments and for people to start getting familiar with it now. Soon we will develop a formal version with cost estimates to go to the Co-Op board.