Observations, stories, projects, photos.

In English and Russian.

6 posts tagged

Tech

Superintelligence: The Next Big Thing?

Jan 23, 2016, 0:17

Some fascinating reading this week!

First, the New York Times asking “Why are the corporations hoarding trillions?” where they claim Apple, Google and other giants don't spend or convert their cash, as if expecting something just beyond the horizon. Something that would need an insane amount of money.

Second, Ben Goertzel with “Superintellingence: fears, promises and potentians”, a through and in-depth review and criticism of current literature by Bostrom, Yudkowsky and others.

This struck me as amusing:

As Peter Norvig of Google has noted (personal communication), quite suddenly we’ve gone from outrage at the perceived failure of AI research, to outrage at the perceived possible success of AGI (Artificial General Intelligence) research!

For the last decade, AI research and theory is booming, and it's very hard to ignore the fact that every other app or service now has some form of machine learning or AI-like capabilities, the most common example being Siri on iOS. There are many more examples, like Google Now or various personal assistants like AlfaSense, X.ai or even something as simple as a blog layout system.

Very excited to see the progress.

Books   Tech

When Xcode goes bad

Oct 6, 2014, 23:19

Sometimes Xcode would not compile your project in its latest state. This is most obvious when you move files around: rename images or remove .xib files. Your app would crash randomly for some weird reason, because Xcode is still using that .xib file you just deleted and tries to initialize it.

In order to fix these caching kind of bugs in Xcode, you have to go to the Derived Data folder and delete everything that's inside (but not the folder itself). In Finder, just use Go ? Go to Folder... and enter:

~/Library/Developer/Xcode/DerivedData

Close Xcode and clear this folder. That's it, after reindexing your project will be good to go.

P.S. This happened most often with Xcode 5, now that 6 is out, maybe that's fixed, but it's still useful to remember that just in case.

Tech

A scroll view instead of a table view

Jul 4, 2014, 0:32

One of the projects I'm working on at the moment is Workie, an offline viewer for Behance, Dribbble and 500px projects. The Behance project view looks like this:

Behance projects are basically a bunch of image and HTML blocks with an occasional video embed. As they are rectilinear and are usually just stacked on top of each other, at first I started to use a UITableView to display the blocks. It worked... okay, until it didn't.

Image blocks are pretty large images, and I used every trick in the book: height caching, background loading and whatnot to speed things up. Performance wasn't choppy per se, but perceptibly slow because of the cell reuse. When a user scrolls down a UITableView, new cells get allocated right at the time when they are about to appear from behind the edge of the screen. If the cell contained a fairly large image, I had to show it empty, even if its height had cached: the system just had no time to load the image up. Since most cells were scrolled into view for the first time, the images weren't in the memory cache, and it also took time to load them from disk. All in all, not a pleasant experience.

As an experiment, I decided to try replacing the table view with a simplistic UIScrollView which got filled with all the project modules (that's what they call them at Behance) dynamically.

Now my cell initialization looked like this, and all the magic was done behind the scenes in the class implementation:

BehanceProjectCell *cell = [cv dequeueReusableCellWithReuseIdentifier:kWPBehanceProjectCellReuseIdentifier forIndexPath:indexPath];
dispatch_async(dispatch_get_main_queue(), ^{
    cell.projectToOpen = project;
});
return cell;

The dispatch_async is necessary because most of the heavy lifting is done in the -setProjectToOpen: method, and we want to return the cell immediately, before it finishes. This buys a lot of time, which is usually enough to load the images and lay out the cell before it ever scrolls into view. Most of the time I see the cells already fully laid out when they scroll out. It's much better than watching a UITableView struggle with loading its cells one by one.

The projectToOpen is an instance of a class that has a modules array, each module having a type enum property. If it's an image, we load an image, if it's an HTML or an embed, we fire up a UIWebView.

The scroll view uses Auto Layout, but we want to do most of the things asynchronously. How to do that, if each module needs to have the previous one in place to set its constraints? If even one constraint is missing, the scroll view will collapse: for Auto Layout to work, we need consistent “pressure” from top to bottom and from side to side. So, how do we achieve that?

We set the basic constraints in the -initWithFrame: method of the UICollectionViewCell (which the scroll view is a part of), pinning the scroll view to the sides of the contentView, which is a container view of a UICollectionViewCell. These constraints will remain in place through the whole life of the cell. The scroll view width is constant and it is centered in the middle of the container view horizontally.

[self.contentView addConstraints:[NSLayoutConstraint constraintsWithVisualFormat:@"H:|-(>=0)-[scrollview(==width)]-(>=0)-|" options:kNilOptions metrics:metrics views:views]];
[self.contentView addConstraints:[NSLayoutConstraint constraintsWithVisualFormat:@"V:|[scrollview]|" options:kNilOptions metrics:nil views:views]];
[self.contentView addConstraint:[NSLayoutConstraint constraintWithItem:self.scrollView attribute:NSLayoutAttributeCenterX relatedBy:NSLayoutRelationEqual toItem:self.contentView attribute:NSLayoutAttributeCenterX multiplier:1.0f constant:0.0f]];

In the -initWithFrame: method, we also add an observer to the scrollView's contentSize property. This is necessary because of the asynchronous nature of loading the images and slow UIWebView HTML load times. At any time, we do not know whether all the images or web content have loaded or not, or if they ever will.

[self.scrollView addObserver:self forKeyPath:@"contentSize" options:NSKeyValueObservingOptionNew context:NULL];

Don't forget to deregister your KVO observer after you're done with it, otherwise the system will complain.

[self.scrollView removeObserver:self forKeyPath:@"contentSize"];

I forgot to say that Behance projects may have a background image, and it can be either tiling or not. Designers use background images a lot to display content while the project image module is just a transparent placeholder. So it is critical to give as much space to non-tiling background images as necessary, not depending on the size of project modules. In the -observeValueForKeyPath:ofObject:change:context: method we observe the contentSize property changes and adjust scroll view insets appropriately, so that the background image fits without clipping. The background image, if it isn't tiling, is a layer added to the scroll view sublayer hierarchy. There is only one background layer, an it's got a witty name “bgLayer”.

for (CALayer *layer in [self.scrollView.layer sublayers]) {
    if ([[layer name] isEqualToString:@"bgLayer"]) {
        CGSize newContentSize = [[change objectForKey:NSKeyValueChangeNewKey] CGSizeValue];
        if (newContentSize.height < layer.bounds.size.height) {
            [self.scrollView setContentInset:UIEdgeInsetsMake(0.0f, 0.0f, layer.bounds.size.height - newContentSize.height, 0.0f)];
        }
        break;
    }
}

Then comes the fun part, the -setProjectToOpen: method. All those dispatch_async look weird, we're already on the main queue, but believe me, I've tested it and it really helps to chop the process into smaller chunks, and the cells work really fast even with large images.

First off, we're setting the background color (not listed), and if there's a background image, spin off its execution to another function. Background images are huge, up to 6000px in height and more, so it's necessary to do that in another place.

After that, in a separate block we create all the views that we'll be using through the lifecycle of the cell until it is reused. Based on the type of the module, we create a UIImageView or a UIWebView, tag it for future reference—the very next function will retrieve them by tags—and after each view is created, we call another function to configure it, also asynchronously. It's all on the main thread anyway, because we're working with UIViews, but it is split into smaller parts, which don't block the UI. One of the reasons for that is that, for example, creating several UIWebViews takes about 70% of total method execution time according to Instruments. A UIWebView is one of the slowest components, yet we're stuck with using it because of HTML content and embeds.

It is necessary to create views in order before starting configuring them because each view depends on the previous one to “latch” to it with a constraint. If we don't create all the constraints, the scroll view will collapse, even if all the other constraints are in place.

if (_projectToOpen.backgroundImage) {
    dispatch_async(dispatch_get_main_queue(), ^{
        [self setBackgroundImageForProject:projectToOpen];
    });
}

dispatch_async(dispatch_get_main_queue(), ^{
    for (int i = 0; i < [projectToOpen.modules count]; i++) {
        BehanceModule *module = projectToOpen.modules[i];
        if (module.type == BehanceModuleTypeImage) {
            UIImageView *imageView = [[UIImageView alloc] init];
            imageView.tag = kWPModuleTag + i;
            [self.scrollView addSubview:imageView];
        } else {
            UIWebView *webView = [[UIWebView alloc] init];
            webView.tag = kWPModuleTag + i;
            [self.scrollView addSubview:webView];
        }
        dispatch_async(dispatch_get_main_queue(), ^{
            [self configureModuleAtIndex:i withProject:projectToOpen];
        });
    }
});

The -configureModuleAtIndex:withProject: is a bit longer method than that, mostly because of different logic for images and web views. Still, algorithmically these parts are the same.

First of all, we get the previously created views by their tags. kWPModuleTag is a constant with a semi-random number, with which we start our tag count. This is in order to avoid tag clashing with something else.

BehanceModule *module = projectToOpen.modules[i];
UIView *tempView = [self.scrollView viewWithTag:kWPModuleTag + i];

Establishing some metrics beforehand is also useful as they are used by all the modules. I have found out that even if you miss a scroll view constraint by a half-point, the scroll view will become wobbly: you'll be able to scroll it sideways about 50 points, though it should not be scrollable in a particular direction. So for margins, I use floorf() and ceilf() to even out this half-point difference. Module width is constant, just like project width. Still, the calculations everywhere are uniform and it wouldn't be difficult to implement a dynamic width scroll view with dynamic width modules.

NSDictionary *metrics = @{ @"width": [NSNumber numberWithInt:kWPBehanceModuleWidth], @"lmargin": [NSNumber numberWithFloat:floorf((kWPBehanceProjectWidth - kWPBehanceModuleWidth) / 2.0f)], @"rmargin": [NSNumber numberWithFloat:ceilf((kWPBehanceProjectWidth - kWPBehanceModuleWidth) / 2.0f)] };

For each module I do five things. The image view here is just an example, the algorithm is the same.

  1. Add a fixed width constraint with margins and a fixed height of 0. We will change the height dynamically later, when the image loads. Until then, the height constraint is necessary to push scroll view contentSize, even though it's zero.
  2. If the view is the first module (its index is 0), add a top margin constraint to the top of the scroll view. The project's top margin is customized and is supplied earlier by the Behance API.
  3. This is the critical part. If the view is in the middle (not the first and not the last), we need to add a constraint from the view to the previous module, which should exist because we made sure to create it beforehand. This inter-module spacing is also customized and retrieved earlier from the Behance API.
  4. Final step for setting up the constraints, if the view is the last one, add a bottom margin constraint to the bottom of the scroll view, which is equal to inter-module spacing. The value of this constraint is not important, because we add ample space to contain the background image (if there is any) in the -observeValueForKeyPath:ofObject:change:context: method.
  5. All preliminary constraints are set up, start loading the data. For a UIImageView, it's an image, asynchronously loaded from the disk or memory cache and set with a completion handler; for a UIWebView, its a dynamically constructed HTML string: Behance API supplies background, paragraph and title styles, which I compile into a static CSS and use in a template with the HTML content of the module.
NSDictionary *views = @{ @"image": imageView };
[self.scrollView addConstraints:[NSLayoutConstraint constraintsWithVisualFormat:@"H:|-(lmargin)-[image(==width)]-(rmargin)-|" options:kNilOptions metrics:metrics views:views]];
[self.scrollView addConstraint:[NSLayoutConstraint constraintWithItem:imageView attribute:NSLayoutAttributeHeight relatedBy:NSLayoutRelationEqual toItem:nil attribute:NSLayoutAttributeNotAnAttribute multiplier:1.0f constant:0.0f]];

// If the imageview is first module, add top margin constraint to scrollview top.
if (i == 0) {
    [self.scrollView addConstraint:[NSLayoutConstraint constraintWithItem:imageView attribute:NSLayoutAttributeTop relatedBy:NSLayoutRelationEqual toItem:self.scrollView attribute:NSLayoutAttributeTop multiplier:1.0f constant:topMargin]];
} else {
    // The imageview is in the middle or in the end, add constraint for previous view spacing, if it exists (it should)
    UIView *previousView = [self.scrollView viewWithTag:(kWPModuleTag + i - 1)];
    if (previousView) {
        [self.scrollView addConstraint:[NSLayoutConstraint constraintWithItem:imageView attribute:NSLayoutAttributeTop relatedBy:NSLayoutRelationEqual toItem:previousView attribute:NSLayoutAttributeBottom multiplier:1.0f constant:moduleSpacing]];
    }
}
// If the imageview is last module, also add bottom margin constraint to scrollview bottom.
if (i == ([projectToOpen.modules count] - 1)) {
    [self.scrollView addConstraint:[NSLayoutConstraint constraintWithItem:imageView attribute:NSLayoutAttributeBottom relatedBy:NSLayoutRelationEqual toItem:self.scrollView attribute:NSLayoutAttributeBottom multiplier:1.0f constant:-moduleSpacing]];
}

At this stage, even if nothing at all loads, we have a perfectly set up scroll view with Auto Layout. It consists of zero-height modules, but that's about to change.

When an image finishes loading from cache, it calls a completion block. I supply a __weak image view in it and check if it's still relevant when the image is loaded. After that, I iterate over the constraints and set the height constraint to a new value, if the image is of predefined width, if it isn't, I scale the image and set its width, height and margins anew. After that I call [self.scrollView setNeedsLayout] and get a notification in my -observeValueForKeyPath:ofObject:change:context: method when the scroll view's content size property changes, where I can adjust the bottom margin to accommodate the background image.

For a web view the process is the same, save that I'm using a delegate, not a completion block. I get the proper height of the web view like so:

CGFloat jsHeight = [[webView stringByEvaluatingJavaScriptFromString:@"document.height"] floatValue];

Again, that looks weird, but it works reliably even when the constraints are contradicting. For example, if I used something along the lines of intrinsicSize or sizeToFit, I would get zero — the value of the constraint that limits the web view's size. By using the JS contraption, I get true height with width being constrained at the same time.

And that's it. After all the content has been loaded, the scroll view now has all the necessary constraints to display all the images and HTML content that it needs, and it does, and it works surprisingly quick.

P.S. AFNetworking doesn't fold lines, and I won't too.

iOS   Tech

Migrating to FastMail

May 7, 2014, 0:27

TL;DR

Procrastinating on configuring a private email server on Amazon (Postfix, Dovecot, SSL, ClamAV, SpamAssassin): 6 months.
Moving 70 000 emails from 6 accounts (3 Google Apps, 1 Gmail, 2 other), reconfiguring MX, DKIM and SPF for own domains: 1 evening.

I wanted to get off Gmail for a long time. The first move was to Google Apps, it worked pretty well, but it was the same server underneath. I almost never use the web interface, so IMAP functionality is pretty important. Gmail's is... well... let's say, peculiar at least.

Recently Gmail IMAP really got worse, not hypothetically. Deleted spam messages started popping back again, a few Sent folders appeared for no apparent reason, all while I was using the same client. It wasn't frustrating per se, but I got tired of that. Also using four Gmail accounts at the same time (3 Google Apps for my own domains and 1 Gmail account) is not very convenient when there are a ton of labels in each one.

The solution was to unite all these accounts and create a unified inbox, which would be easy to navigate and fast to use. First, I wanted to just larch the messages from my Gmail and other Google Apps accounts into my main Google Apps account, the one on this domain. For some reason this did not seem like a good idea: if something went wrong, it would be pretty difficult to decouple the accounts and put everything back together. I'm a sysadmin, so Branson's famous “screw it, let's do it” isn't something I do often. You don't rush things when there's a possibility that you'll have hundreds of people unable to work for a few hours. Better spend two days of your own time on a problem and do everything smoothly. Well, you get my drift.

In the end I settled on rolling my own server, there are plenty of manuals around. Keywords are Postfix and Dovecot, those are two key components to the setup: Postfix gets mail and Dovecot lets you read it. All the other parts are more or less optional and depend on personal preference. I compiled my own best config and set out to do this a little later: I just left my job and had a few short months ahead to learn a new occupation.

Fast forward six months, the private server is still on my monthly to-do list, but I don't have that much free time to spend a couple of days configuring it. Gmail, after all, isn't the worst mail server. A few days ago I stumbled upon a discussion (which is private, sorry guys), where someone mentioned FastMail. I read up on them, and they have a pretty interesting history. Just recently the company bought itself back from Opera so that they could continue running their email, clearly they care for their customers a lot.

Actually, the FastMail/Gmail thing reminded me of App.net, which is, in a nutshell, a user-supported completely ad-free version of Twitter. It's the same with FastMail: because it is not a free service, they can really care for their customers, and the customers love them back. I know I do. Just using their email for a few hours shows how much they care, and it gets me every time.

But let's get back to the point. FastMail offers a 60-day trial, virtual domains and IMAP migrations, all of which are necessary for me to a) try out their service b) get my domains over to them and c) get my email into FastMail. All of which I did, spending just a couple of (active) hours. Finding a service that is actually user-centered, compared to Google, and offers custom everything out of the box instantly seemed a no-brainer for me to run with that instead of spending my time on rolling my own solution.

In the end, that is a question of priorities. For me, having full control over my email is of less priority than having a single mailbox that lets me write from any associated email address and also collects email from all my other accounts that I don't actively use. I'm not afraid of someone screening my email because I'm not a criminal. Still, FastMail let me make another step to independence: now instead of six accounts I only have three, one being FastMail and actively used and others just forwarding email just like they were to Gmail before.

Next steps: keep a synced and backed up local copy of the whole mailbox with offlineimap, which can easily be uploaded to a private server if I ever have one, because there's only one active account now and because I'm using my own domain instead of a @gmail.com address. Just set up the server, upload email, change MX and off you go.

You know why it's called FastMail? It does work really fast.

Tech

Poor man's CPU stress test

Feb 9, 2013, 22:42

Just a few days ago I had to stress test the CPU on a machine with nothing installed but bare Linux and no internet access.

I googled a bit, found a couple bc expressions and modified one that calculates 5000 characters of pi like this, so that it runs indefinitely:

$ while true; do time echo "scale 5000; a(1)*4" | bc -l; done

This is of course non-parallel, so run as many of these as you have cores to get almost 100% CPU load:

It's also a great benchmark for CPU performance, just record the time and compare it to friends' results :)

Tech

Logon to Windows remotely as local user

Feb 9, 2013, 1:11

A friend showed me today how to conveniently logon to a local or remote machine as its local user, even if it's in a domain. The trick is to prefix the username (local to the remote machine) with a “.\”, which (obviously) means “local domain”.

So, to logon to a remote machine via RDP as the machine's local Administrator, enter this in the user prompt:

.\Administrator

The following screenshot is taken on a machine joined to a domain. Notice how “.\” is interpreted as a local domain, just below the password box. Observe:

Local domain: .\

The best thing is you can also use this for entering any remote user's credentials on your own machine. There's a catch: when you're presented with a credentials prompt on your machine and enter the remote username prefixed with “.\”, Windows shows the name of your local machine below the password box. When you actually use the credentials to login to the remote machine, it interpets “.\” as its own “local domain”, and you're logged in successfully. So don't let your OS fool you :)

I've said quite a few bad words while trying to find out what a domain machine's (often cryptic) name is to logon as a local Administrator. It's probably a basic technique taught for MCSA and suchlike, but neat nevertheless, isn't it?

Tech