Sharing Files With Pythonista

20. February 2021 · Categories: Software

I recently had the problem to easily share files generated by Pythonista on iOS. The easiest way is to show the share sheet with the file afterwards, and this turns out to be trivial, once I figured out the logic.

The trick is that files can be shared as URLs, you just need to convert them first to an absolute path, and then to an URI so that the API understands them properly. Much more comfortable than searching for the generated file and then invoking the share sheet manually.

The Second Corona Wave

17. November 2020 · Categories: Politics

It is highly disconcerting to see the corona numbers rising again among almost the entire Western world, even though our understanding of it has much improved. I see a few areas where the successes of the Pacific Rim are not copied, and so a lot of unneeded harm is done:

Aerosol transmission is still denied as important, even among some leading public figures. The details can be found in this FAQ. With aerosols, not only is social distancing important, but we need to put additional emphasis on ventilation, and on mask wearing. Indoor spaces where we have high incidence rates or longer stay times should be avoided. Avoid meeting people indoors. At home, it means that wearing masks and keeping distance when you have guests will not compensate for sharing a badly ventilated room for several hours. And if you want to have a larger gathering, you need to realize that it can only be safe if everyone is virus free. So guests need to isolate for two weeks beforehand, and should for a week afterwards as well, to prevent a possible infection from spreading wide.

Contact tracing is failing badly. We need to accept that we cannot do contact tracing without keeping detailed records, and we need to accept that contact tracing is privacy invasive, and that the solution is to punish people abusing that information, and to effectively control access to it, not to anonymize the data so early that it becomes ineffective. We need to keep records where and when people are in indoor spaces, or other gatherings. This can be done with QR codes, and keeping a history on your phone that is only shared when you are infected. We also need to accept that transmission is a lot via super spreading, and that there are quite a lot of asymptomatic spreaders. Thus we will be much more successful if we not only trace the people potentially infected by our case, but also figure out where our case got infected. A super spreader might be asymptomatic, but at least one of the infected should show symptoms, and so allow us to find them. And lastly we need to be aware of our tracing capacity, and keep cases low enough to prevent it to be overwhelmed. This includes having around 200 spare tests per identified case to quickly test identified contacts.

Testing and Tracing turnaround times must be kept below 2 days. We need to test our case, contact the contacts, and have them tested as well within this time to be able to isolate effectively before you become infectious.

Elimination would have been overall a lot cheaper than the half measures taken in Europe and the US. You are comparing three weeks with no output to a year with 15% output reduction and permanent stress. It would have been best to announce an October lockdown already in June, lasting for three weeks, with time to prepare. And the lockdown then so stringent to eliminate all cases.

Compliance is the large issue with lockdowns, there are always some who believe that they can bend the rules. But I believe it is much more an issue with an unending, year long, lockdown lite to keep numbers just in check. For a mere three weeks, with enough time to prepare, compliance is a much better sell.

Apple M1

12. November 2020 · Categories: Apple

The new Apple silicon for the Mac starts with a chip that provides roughly the same performance as the top end Intel chip in MacBooks, while consuming so little power that the new Air no longer has a fan.

The transition will likely go smoothly. As we no longer have self modifying code, translation of x64 code to ARM is now feasible at reasonable performance, and the chip has enough headroom for the losses incurred.

The new Mac mini is a missed opportunity: with the same underlying architecture as Apple TV, it would have been nice to see a home focused variant, that includes Apple TV, and serves as a home server as well as a WiFi router.

The architecture is now tightly integrated, with shared memory for all blocks. I wonder how this will translate to higher performance configurations. Either you go for really large chips to keep the integration, or you need multiple chips with corresponding problems in synchronizing memory. I suspect that large chips are now feasible, with dozens units you can plan for disabling a few to avoid defects, and the caches could be 16 way, allowing you to disable one way where you find defects.

With any multi chip solution you have limited bandwidth to address cache coherency issues, and congestion accessing the shared memory. This is why massively parallel computers put their emphasis on fast links between nodes, and not on a fast, shared memory.

iPhone 12

09. November 2020 · Categories: Apple

The iPhone now has four variants for the mainstay 12: 12 mini, 12, 12 Pro and 12 Pro Max. The technical differences are relatively small, and much of the consideration boils down to the phone size you prefer.

The Pro’s have lidar, 6 GB RAM, and three lenses. On the normal Pro the extra lens is 52mm, on the Pro Max 65mm with better stabilization. The Pro Max also has a better 26mm lens, adding roughly half a stop more light gathering. This gives three levels of camera performance, with the base level already great for most casual shots. If you want more, you might want to consider something like the RX100 VII, which is not too heavy, and complements the phone quite nicely.

Unfortunately, the Pro models have worse materials in their design: the steel bands add to the weight, non glossy backs make them more difficult to hold, and the color choices are boring.

The main difference really is the size: the Max is too large for one handed operation, the middle size is kind of OK for men, and the mini is the best one. Without the margins, the mini has 15% more screen than the 2nd gen SE, and this is sufficient, especially if you have an iPad as well.

Personally, I like having the 52mm enough that I am willing to compromise on size to get the Pro, but it is not really what I want: camera system of the Pro, size of the mini, but about 2-3mm thicker to accommodate a larger battery and to hide the camera bump.

PS: After a month the 12 Pro still feels too large and heavy. Just enough to irritate me, not enough to dump the phone. It affirms that the phone I really want is the 12 Pro mini outlined above.

Privacy with Contact Tracing is an Illusion

07. May 2020 · Categories: Politics

With everyone looking at Bluetooth to enable a privacy preserving manner of contact tracing, it is time to acknowledge that it will not work. There are multiple ways in which Bluetooth contact tracing is insufficient:

Accuracy of contact distance is low with only Bluetooth. On the one side, the distance can be a lot shorter than estimated when something blocks our phones, but not our heads. On the other hand, it can easily penetrate walls and glass, so people separated by such barriers can be seen as contacts, even if there is no way past for the virus.

No context is provided, so we do not know anything about ventilation or interaction between people.

Large adoption is necessary, so we likely cannot avoid forcing people to use the tracing.
To solve these problems, we need to acknowledge that contact tracing is inherently invasive, include rough geo tracking data to help people remembering what they were doing, and use a lot of contact tracers to provide judgement. Acknowledging this allows us to concentrate on the different set of problems we need to solve for tracing anyways:

Is there enough testing capacity? Without it, we run the risk of isolating too many people. Also we will not be able to learn which kind of interactions have a high transmission risk.

Is the data secure? When we allow contact tracers access to invasive information about ourselves, we need to make sure that this will not be abused. We need infrastructure security against hacking, we must have monitoring in place to prevent tracers from abusing the often personal information they are learning, and add laws1 to ensure that nobody will try to use this data for anything else than contact tracing and understanding transmission pathways. It is also important that any of the historical data created from tracing will be thrown away once we have found all the contacts, and only statistics about transmission modes are being kept.

How do we select and train tracers? We need a human factor to also look for the few people that are not using smartphones, and to help with organizing the necessary quarantines. And we need a selection process to filter out people that would violate confidentiality.

Are there safeguards against mission creep? With such an invasive setup, it is much less likely that we will tolerate it for anything but contact tracing. Also ensuring that all the data collection is opt-in, even when using it is mandatory, creates a kill switch that will stop the infrastructure once the crisis is over.

All of these problems are still difficult enough to solve, and the earlier we stop wasting resources chasing after a pipe dream, we will have a proper discussion about to implement actually efficient contact tracing.

  1. Basically stiff penalties plus jail time, and a mandate to throw away data when adding additional uses. A consequence is that we would accept to let murderers go free, instead of using the tracing data. ⏎

Coronavirus Notes

29. March 2020 · Categories: General

Remember that the virus can already be spread by people not yet showing symptoms, or so mild ones that they are not correctly identified. So you need to be careful not to spread it always, even without symptoms.

What everyone needs to do

Keep at least 1.5m distance to other people, preferably 3m. Since the transmission is mostly via droplets from breathing, especially coughs and sneezes, they will fall down within this distance. There was a case where a passenger in a bus infected other people sitting 4.5m away, and also another person entering the bus 30min later, while some people closer by got lucky and were not infected. This clearly demonstrates that you should be taking extra care in enclosed spaces, where longer stay durations will saturate the air more with virus particles.

Regularly wash your hands with soap, or alcohol based disinfectant if not available. The virus can stay viable for up to 72 hours on surfaces, especially steel and plastics, and you can pick it up with your hands and then spread it to your face. Wash them before leaving the house, after coming back home, and often while outside. Remember to open any doors on your way to the hand basin before leaving the house, you don’t want to contaminate them when coming back home. If you are worried about a package being contaminated, you can let it rest somewhere for three days, after which the virus will be deactivated.
Also regular washing will irritate your skin, so do not forget to care for your hands. An alternative could be water with ozone mixed in.

Do not touch your face. Easier said that done. But make sure when you are outside your home. It helps to clean your face before leaving the house, as it reduces irritations that might trigger you to touch your face.

Wear a simple face mask. Masks do not protect you very much, but even a mask made of cotton will reduce the droplets you are spreading. They are most helpful to reduce community spread. Cotton masks do not work for long, they start to lose effectiveness already after half an hour, and should be replaced after one hour. For some background, see this Medium post by Sui Hang.

Avoid close contact with many different people. You might need to see some people to get through a longer lockdown, but we ideally want to be split into a lot of small, separate groups to prevent the virus spreading far. Putting together a small friends group, where everyone agrees to see only people within this group is pretty reasonable. But keep in mind that this needs to include everyone sharing a single home, so two friends with family would already be a group of 8, and making them larger is not really helpful.

What we need to do as a society

Organize widespread testing. We probably need to test around 100 people for every fresh case to find everyone infected, and to isolate them. This will mean 10 million or more tests daily for the entire world to control it. Without it we can never be sure who is clean of the virus, and cannot remove the lockdown.

Much improved contact tracing. We need to use the tracing abilities a modern phone can give us to quickly locate contacts, without needing a lot of man power. We can use both Bluetooth for automated contact lists, or GPS for time/location matching.

Closing down public transport, because it causes many people to be in close contact. We should be able to provide fast ebikes as an alternative.

Support people that cannot do their jobs because of the shutdown. This is a temporary demand shock, which does not change the underlying viability of the business once the lockdown is no longer needed. We have enough man power to keep everyone fed, so we can afford to keep them fed well. This will also mean that we should freeze payments for capital that temporarily cannot utilized because of lockdowns. Not only the businesses currently not having any demand, but also the capital owner need to make sacrifices to get through this situation.

Why did it get so bad?

We lost many opportunities to limit the spread of the virus thanks to incompetence and also poisonous politics.

When the virus started last year in Wuhan, local doctors sounded the alarm, but the communist party violently suppressed it, so that when they finally admitted to it, we probably already had 1000s infected.

The WHO was way too late in acknowledging that there is human to human transmission. Taiwan was already informing the WHO about it on 31.12.2019, while WHO still repeated China’s insistence that there wasn’t on 14.1.2020. And the poisonous way China is treating Taiwan also meant that Taiwan could not warn the world.

Then all countries not having experienced SARS did not understand the severity of the situation, even with the example of exponential growth and the severe lockdown in place in China. They for way too long considered it just a bad flu, didn’t grasp that a doubling time of 3 days means 1000 times more cases within a month, and didn’t take China serious, seeing the lockdown as a crude tool deployed by a backwater, instead of an indication of how serious COVID-19 is.

They also did (and still do) not grasp that a disease that is quite similar to influenza can spread quite a while undetected unless you are testing very aggressively. So all travel warnings came way too late.

The endgame

Mortality seems to rise with age, leaving people below 50 with a worst case mortality of 1%, and maybe 5% with permanent health damage. This is still awful, but it is low enough that societies will not collapse.

Also there is the paradox of Japan: Even though the country does not have a lockdown, limited testing and contact tracing, corona is not tearing though the country. About the only precaution seems to be widespread wearing of face masks.

The Retina Monitor Transition

28. September 2019 · Categories: General

Since the introduction of the iPhone 4, high resolution screens have become the norm in mobile, and now every mobile computer Apple makes has a Retina screen, after the retirement of the old MacBook Air. Now the transition is moving on to external displays, with intermediate steps of 5K (iMac, 5120 by 2880) and 6K (Pro Display XDR, 6016 by 3384), but we are not yet there with affordable 8K displays.

As can be seen in the following table, data rates increase quite quickly with larger displays. This is with 8 bits per color and 60 Hz.

Resolution Data
3840 x 2048 11.3 Gbit/s
5120 x 2880 21.2 Gbit/s
6016 x 3384 29.3 Gbit/s
ditto, 10bpc 36.6 Gbit/s
7680 x 3200 35.4 Gbit/s
7680 x 4096 45.3 Gbit/s

DisplayPort is the standard technology to connect monitors, and it supports 26 Gbit/s since 1.3, and version 2.0 from June 2019 will extend that to up to 77 Gbit/s, with options for 37 Gbit/s and 50 Gbit/s.

We see in this that all the new monitor options from Apple make full use of step changes in the available bandwidth, to provide the most pixels that one can send over a given monitor cable. Apple does not like compression, as it could introduce artefacts. The alternative is DSC (Display Stream Compression), which has been developed with video and text display in mind, and should only produce small, imperceptible losses.

So my expectation is that any computer that can support the 6K 10bpc resolution will also allow a Retina 38″ display, while for 8K displays we will likely see compression as an intermediate step until 50 Gbit/s bandwidth has matured.

Retina class large displays are used with a working distance that higher resolution is only marginally appreciated. Probably a 1.5x increase would be sufficient, but could lead to issues because of scaling artefacts with old software. Many people might see more benefit from switching over to a higher color depth instead, and it could well be that 8K 10bpc will become the new standard, combining two nice improvements into one compelling upgrade story.

The last improvement vector are refresh rates. While 60 Hz are enough for a fluent display, we have multiple components in the display chain that can lead to additional lag. This includes buffering, where we calculate the image with information at the start of the frame, but have the entire image only ready at the end, the encoding delay for transferring the screen buffer info from the computer to the display, and also a possible quantisation effect when the refresh cycles of the computer and display are not synchronised. The proper way to reduce this would be proper synchronisation between both sides, but this requires full stack optimalization:

  • Sync the frame starts between computer and monitor

  • Improve you graphics pipeline to be able to do deadline scheduling, and start to calculate your next frame just late enough that it becomes available just before you switch over to that output buffer.

Even though it would conserve power, the difficulty of knowing the calculation complexity of a scene beforehand leads most games to just calculate frames as fast a possible, and then throw away extra frames. This is also the reason that some use triple buffering to smooth the output, allowing a small overrun on one frame by taking time from the next one without the user noticing.

A two frame delay is about the optimum to expect, one for the buffer on the computer, one for the frame transfer to the screen, giving us 33ms at 60 Hz. For optimal reactions you would want to half it, giving another doubling of the transfer rate. On the other hand, a delay of 50ms compares well to human reaction times of around 200ms, but it is at the margin if are trying to lock onto moving objects.

Folding Phones

24. February 2019 · Categories: General

Now that we are seeing the first folding smartphones coming out, I feel fundamentally pessimistic about them. Unfolding will always take an extra second or two, and that is problematic for quick checks.

As long as the phone remains our primary communication device, we will value quick access a lot. So either another device must replace the phone. The watch and AR are the best candidates for now, but both are still far away from being capable devices for responding. Or the folding phone must work very well both folded and unfolded. This is a challenge, with the extra weight and space required, and with the screen the largest battery drain. I am afraid that folding phones will need to add too much bulk to stay competitive with normal phones, especially now that the standard size has become large enough to comfortably read even longish texts. The declining sales of the iPad mini give a strong indication that the expanded state would not be valued enough to justify costs.

I am much more optimistic about folding tablets. These are already devices that are used for longer sessions so the folding will be much more tolerable, and it provides a real benefit in making tablets more portable. We will have to wait a bit though until the technology has caught up to allow large, durable, folding screens with good enough power consumption.

A Cortex-M Memory Bug

11. February 2019 · Categories: Software

I came across an interesting bug recently. Suddenly my firmware would start crashing, and in the debugger I see that it calls a function with wrong parameters. The offending code:

When the problem happens, var is actually 0. It turns out that the load of the variable address returned the value stored just before. On Arm, this is a PC relative load LDR R3, [PC, offset]. How can something so fundamental go wrong?

The behavior under the debugger is odd. When I single step through the code, there is no problem. I put a dynamic breakpoint after the load, triggering when a bad register value is detected. Still never triggered. I hit go, and just a few iterations later I hit the problem.

The code is executed just after the processor wakes from sleep, and the problem only happens after waking from deep sleep, but not from normal sleep. After deep sleep I need to switch from the default clock to my desired clock. The problem does not happen immediately, there are a dozens others instructions correctly executed after waking up before the read error occurs. What can cause this?

I use an STM32 processor, and run it using the HSI48 clock used for the USB peripheral. The processor has a clock recovery system (CRS) which can automatically trim HSI48 to be precisely 48 MHz. I have selected the internal RTC as the trimming reference. Now while the CPU is in deep sleep, all clocks apart from the RTC are stopped. When waking up, the CRS resumes counting once we are back on the HSI48. But the wakeup is caused be the same RTC, so the RTC clock events as seen by the CRS are systematically delayed, and so it decides to trim the HSI48 faster, and faster. In the end the processor is clocked just a tiny bit too fast for its internal flash memory, so reads go wrong.

To avoid this runaway, I updated the deep sleep code to reset the trim value after waking up:

So when your flash reads start to fail, remember to check the clocks. They might be trimmed outside specifications.

iOS 11 Dock

30. September 2017 · Categories: Apple

I find the implementation of the dock in iOS 11 annoying in a few small ways:

When you access the task switcher, tapping onto the background takes you to your home screen. But if you leave via the home button, you get back to your previous screen, no matter whether it is an app or the home screen. Because the task switcher also contains the quick access buttons, it means extra mental load when I just want to change a setting. Tapping on the background should always return you to your previous screen. If you want quick access to the home screen from the switcher, it should be a separate, clearly indicated area. Maybe it would be a small home strip above the spaces display.

It is too easy to accidentally kill an app. The flick to destroy gesture is very satisfying, but too easy to trigger when you grab an app and are just playing with the animation. Something like the trash can would be great to improve this.

To add an app not in the dock to side by side, you start dragging with one finger on the home screen, then open the app switcher, and then drag the app to the correct space. If you are not fast enough, you often open the wrong space. Instead I suggest that dragging an icon to the right or left of the dock should place the app into side by side with the most recently opened app. Dragging it there will provide feedback in the form of an animation, just like dragging an icon from the dock to the sides while an app is active. If you keep hovering a bit, the most recent app will come back to the foreground. The area to the side is large enough that accidental activation should not be an issue.

There are also a few bugs. Quite often the task switcher view forgets to blur the background. Also in the home screen, if you show your widgets, go back to the apps, and then rotate the screen, the docks stays in place. I am a bit worried about the last bug, because it seems like a problem that anyone using the iPad handheld would have quickly spotted.

« Older articles
Newer articles »