deGoogling, part 1

I’m engaging in a slow effort to decouple from Google.

My first step was to decouple email, calendar, and contacts. For a long time I’ve been using Fastmail as a mail server for my non–Gmail domains, and then fetching this mail into Gmail. Before deciding to reverse this flow, I briefly considered ProtonMail and Mailbox.org. While I appreciate the data residency and security of ProtonMail, it seems that the added security creates some connectivity challenges. As for Mailbox.org, there were no compelling features that motivated me to switch from Fastmail.

Fastmail provides direct integration with Gmail to import mail, contacts, and calendars. Before doing this, I first disabled Gmail’s email import from my Fastmail account. When I first attempted to import my Gmail content into Fastmail, I was surprised by the storage estimate. Google reported that Gmail was using about 4GB of my Google storage, and this matched the size of email data when I downloaded all of my email from Google Takeout. However, Fastmail reported that there was an estimated 116GB of email that would be downloaded from Google using IMAP. This was quite disconcerting to me, partly because Google indicates that they may throttle IMAP downloads to 2.5GB per day, and partly because Fastmail charges based on my storage usage.

The Fastmail import from Gmail may create duplicate email messages for any emails that have multiple tags, so my first thought was to ensure that I had no emails with multiple tags. However, this did not affect Fastmail’s download estimate. My second thought was to sift through old archived emails that I no longer needed, and while this helped, it did not help significantly.

After working with Fastmail support, it turns out that the IMAP import size estimate is based on the entire storage usage that Google reports, which includes Google Drive: and I currently have over 100GB of data on my Google Drive (stay tuned). After crossing my fingers, I went ahead and imported my email. Google’s 4GB of email was fully imported in just under an hour, after which Fastmail reported that I had about 4GB of email.

When I imported my own email, for some reason the Gmail Sent and Spam folders appeared as custom folders within Fastmail, and I had to move the emails to Fastmail’s equivalent folders. However, when I did this for my wife’s email, these two folders transferred seamlessly.

I appreciate that the keyboard shortcuts in Fastmail’s web interface generally match those of Gmail. I also appreciate how easy Fastmail makes it to configure Apple devices. So far the experience has been relatively seamless for me, and I haven’t noticed any problems at all with my contacts or my calendar.

Complex

I’ve joked for awhile that EDR and similar systems like CrowdStrike or Carbon Black would become Skynet. Or, more likely, a tool of international espionage and cyber–warfare. It doesn’t feel good to be vindicated. (Now imagine someone accomplishing this with a major browser or password manager application.) Complex and highly interconnected systems are difficult to make stable, resilient, or secure; and cannot possibly be made anti-fragile. (This is a lesser reason why I miss my old pickup truck.) I’m not very excited about Kubernetes for the same reason. It’s also partly why I’m not very excited about artificial intelligence; but additionally because analysis of data, whether by machine or by human, does not automatically involve either wisdom or decisiveness (see also Edwin Friedman and Nassim Taleb).

Crossposted on I gotta have my orange juice.

How to bring your team back to the office

A lot of people are offering advice on how to return to the office safely. Here is my evidence-based approach:

Bring your team back to the office.

You should, of course, continue to give people the freedom to work at home if you are able.

I’ve written much more on my other blog stressing that churches must be open for worship, and should do so as normally as possible. But there is also a tremendous benefit to other spheres of life—business, family, education, politics—for people to be face to face with one another. Beyond that, there is actual harm in exercising authority to constrain any of these spheres of life beyond natural limits.

Here is my evidence: Alex Berenson’s Unreported Truths. In addition to that, Edwin Friedman’s A Failure of Nerve offers the appropriate framework for leadership in difficult times: non–anxiety.

reCAPTCHA v3 on Rails

I implemented reCAPTCHA v3 on a Rails site recently. It was very straightforward, and I’m generally pleased with the outcome. Interestingly, the vast majority of spam requests seem to be made without having generated a reCAPTCHA token, suggesting that they are not even loading JavaScript. This points to a possible poor man’s approach for spam suppression: generate your CSRF token using JavaScript rather than emitting it directly in the form.

For better or worse, it is difficult to operate on the web today without JavaScript.

Version 3 of reCAPTCHA operates by capturing all user site activity, which is a privacy concern, but also allows it to function unobtrusively.

I defined several variables in my environment:

RECAPTCHA_CHECK = true
RECAPTCHA_SITE_KEY = 'xyz'
RECAPTCHA_SECRET_KEY = 'xyz'

I load the reCAPTCHA script on secondary pages of my signup site, but not the root page and not any internal pages:

<% if RECAPTCHA_CHECK -%>
  <script src="https://www.google.com/recaptcha/api.js?render=<%= RECAPTCHA_SITE_KEY %>"></script>
<% end -%>

Then I attach an action to the signup form submission to generate the reCAPTCHA token and include it in the form data (the form is named signup_form and has a hidden input named recaptcha):

<% if RECAPTCHA_CHECK -%>
  // Add reCAPTCHA token on form submission
  $(function() {
    $('#signup_form').submit(function(event) {
      event.preventDefault();
      $('#signup_form').off('submit');
      grecaptcha.ready(function() {
        grecaptcha.execute('<%= RECAPTCHA_SITE_KEY %>', {action: 'submit'}).then(function(token) {
          $('#recaptcha').val(token);
          $('#signup_form').submit();
        });
      });
    });
  });
<% end -%>

This site already has several conditions for trashing a suspicious signup request, with a simple error page that directs visitors to reach out to our administrators by email. I added a new condition to check the reCAPTCHA token:

if RECAPTCHA_CHECK
  # Check reCAPTCHA
  recap_uri = URI.parse('https://www.google.com/recaptcha/api/siteverify')
  recap_params = { secret: RECAPTCHA_SECRET_KEY, response: params[:recaptcha] }
  response = Net::HTTP.post_form(recap_uri, recap_params)
  logger.info "Recaptcha result: " + response.code + " / " + response.body
  response_json = JSON.load(response.body) if response.code == "200"
end

# Silently throttle requests that fail reCAPTCHA
if RECAPTCHA_CHECK && (response.code != "200" || !response_json['success'] || response_json['score'] < 0.5)
  Mailer.exception_notification(Exception.new("reCAPTCHA throttled signup"), params).deliver_now
  redirect_to :action => :thankyou
end

As you can see, for now I am operating with a threshold of 0.5, but I may adjust this over time. In my own testing, I generated a confidence value of 0.9.

Rest

Rest

We considered how stress and self-discipline result in growth and strength, whether that is physical, mental, emotional, or spiritual. However, an important corollary of this is that intervals of rest are needed so that we are able to recover stronger instead of ending up progressively worn down.

From nature and our own experience we can see that this rest needs to happen on several cycles. There is a daily rest (1/3 of our time is spent sleeping), a wise principle of weekly rest (one day out of seven), and a yearly rest (winter, vacations). We could even consider the wisdom of longer cycles of rest (e.g., taking sabbatical every 7 to 11 years as many universities practice for their faculty, and as Intel has done).

These principles apply not only to organic life but also to organizations. While agile principles and techniques do increase team efficiency and productivity, it is a mistake to think that agile’s goal is continuous apparent productivity. There are a number of shatterings of continuous apparent productivity that are necessary to healthy agile product development. It is important to brainstorm, learn, conduct retrospectives, take time to refactor, experiment and evaluate alternatives . . . and also to rest. Paradoxically, all of these ways of taking time to slow down often help to improve your team’s long-term productivity.

Obviously our individual daily, weekly, and annual cycles of rest help with the health of our agile team. But the team itself should also be engaging in rest. There are many possibilities here, including team outings and shared meals, team training, and planning for gap sprints or gap weeks to focus on lighthearted or experimental work (what if I rewrote this in Clojure, Haskell, or Racket). In keeping with the spirit of agile, the team should evaluate its own need for rest and plan appropriate kinds of rest.

Crossposted to I gotta have my orange juice.

Difficulty

Mark Horne writes of strength training:

The rule seems to be that your body adapts so that the most difficult thing you do eventually feels hard to do. As you age this process accelerates. When you give up an activity because it feels hard another one starts to feel hard to do. As your body loses strength you start to avoid tasks and chores that were once easier. You accumulate weakness. In the words of Seneca, “Soft living imposes on us the penalty of debility; we cease to be able to do the things we have long been grudging about doing.”

But this is true not only of your body but also your mind and will and spirit: the hardest thing you do feels hard. This leads us to several helpful insights:

First, it helps us sympathize with others who are experiencing difficulty. It is tempting to despise others who have greater difficulty with smaller challenges compared to yourself. However, this principle allows you to sympathize, since you know that difficulty is relative rather than absolute.

Second, this teaches us that contentment, peace, and joy are not primarily related to our circumstances but to our philosophy and outlook on life. Excluding obvious exceptions such as injustice and extreme hardship, this principle reveals that if you are complaining or anxious in one difficulty, you will still be complaining or anxious in other and even lighter difficulties. Therefore, your work to cultivate contentment, peace, and joy cannot wait; you must find deep roots unrelated to your circumstances. And even in cases of injustice and extreme hardship, this reveals that there is a possible path to contentment, peace, and joy even while you wait on, plead for, and pray for relief.

Third, this also indicates a way to grow in our capacity for work and difficulty. It is helpful simply to recognize that difficulty is relative, since you can cultivate gratitude that you are not experiencing greater difficulty. But this also gives you a tool to expand your capacity: you can periodically subject yourself to greater or artificial difficulty, combined with periods of rest and recovery, in order for your current difficulties to become lighter. In the physical sphere, you increase your capacity with sprint exercises, intervals, and progressive loading. Furthermore, growth in self-discipline and capacity in one sphere of life tends to have a side effect benefit across all of life. It is strangely easier to wake up early and to eat well if you are working hard at strength training; there is a kind of snowball effect to growing in health and strength and capacity.

Finally, all this applies not only to yourself but also to how you can lead others to grow in joy and capacity. As Edwin Friedman writes, “increasing one’s pain threshold for others helps them mature.”

Crossposted to I gotta have my orange juice.