Tag Archives: software development

Web page memory usage: how much is too much?

I have a web application that loads names into a picklist and the question came up: how many names would be too much for the web page to handle? What about 5,000 names, for example?

Modern web browsers have a memory snapshot built in. Just press F12 and there it is. So I fired up my application with over 5,000 names loaded into an array and displayed in a scrolling div. 2.5 MB.

Then I visited Facebook. Logged in, the page reported over 78 MB.

image

Google’s clean-looking home page? Not logged in, 11.2 MB.

image

Twitter? Logged in, 83 MB.

image

This was enough for me to stop worrying about the memory impact of 5,000 names in my web application. With our casual acceptance of multi-MB web pages it is easy to forget that the complete works of Shakespeare in plain text is 5.5 MB and compresses to less than half of that.

Just because you can do something, does not mean you should. Smaller is better and faster, and software bloat of various kinds is responsible for many performance issues.

There are trade-offs though both in time and in performance. Maybe it is better to load just a few names, and retrieve more from the server when needed. It is easy to test such things. Nevertheless, I found it useful to get some perspective on the memory usage of modern web sites.

One .NET: unification of .NET for Windows and .NET Core, Xamarin too

Microsoft’s forking of the .NET development platform into the Windows-only .NET Framework on one side, and the cross-platform .NET Core on the other, has caused considerable confusion. Which should you target? What is the compatibility story? And where does Mono, the older cross-platform .NET fit in? Xamarin, partly based on Mono, is another piece of the puzzle.

Now Microsoft has announced that .NET 5, coming in November 2020, will unify these diverse .NET versions.

“There will be just one .NET going forward, and you will be able to use it to target Windows, Linux, macOS, iOS, Android, tvOS, watchOS and WebAssembly and more,” says Microsoft’s Rich Turner.

image

Following the release of .NET 5.0, the framework will have a major release every November, says Turner, with a long-term support release every two years.

Some other key announcements:

  • CoreCLR (the .NET Core runtime) and Mono will become drop-in replacements for one another.
  • Java interoperability will be available on all platforms.
  • Objective-C and Swift interoperability will be supported on multiple operating systems.
  • CoreFX will be extended to support static compilation of .NET and support for more operating systems.

A note of caution though. Turner says there are a number of issues still to be resolved. There is room for scepticism about how complete this unification will be.

More details in the official announcement here.

Update: having looked at these plans in a little more detail, it is wrong to say that Microsoft is unifying .NET Framework and .NET Core. Rather, Microsoft is saying that .NET Core is the replacement for .NET Framework for new applications whether on Windows or elsewhere. Certain parts of .NET Framework, including WCF, Web Forms, and Windows Workflow, will never be migrated to .NET 5. .NET Framework 4.8 will still be maintained and is recommended for existing applications.

Desktop development: is Electron the answer, or a tragedy?

A few weeks ago InfoQ posted a session by Paul Betts on Desktop Applications in Electron. Betts worked on Slack Desktop, which he says was one of the first Electron apps after the Atom editor. There is a transcript as well as a video (which is great for text-oriented people like myself).

Electron, in case you missed it, is a framework for building desktop applications with Chromium, Google’s open source browser on which Chrome is based, and Node.js. In that it uses web technology for desktop applications, it is a similar concept to older frameworks like Apache Cordova/PhoneGap, though Electron only targets Windows, macOS and Linux, not mobile platforms, and is specific to a particular browser engine and JavaScript runtime.

image

Electron is popular as a quick route to cross-platform desktop applications. It is particularly attractive if you come from a web development background since you can use many of the same libraries and skills.

Betts says:

Electron is a way to build desktop applications that run on Mac and Linux and Windows PCs using web technologies. So we don’t have to use things like Cocoa or WPF or Windows Forms; these things from the 90s. We can use web technology and reuse a lot of the pieces we’ve used to build our websites, to build desktop applications. And that’s really cool because it means that we can do interesting desktop-y things like, open users’ files and documents and stuff like that, and show notifications and kind of do things that desktop apps can do. But we can do them in less than the bazillion years it will take you to write WPF and Coco apps. So that’s cool.

There are many helpful tips in this session, but the comment posted above gave me pause for thought. You can get excellent results from Electron: look no further than Visual Studio Code which in just a few years (first release was April 2015) has become one of the most popular development tools of all time.

At the same time, I am reluctant to dismiss native code desktop development as yesterday’s thing. John Gruber articulates the problem in his piece about Electron and the decline of native apps.

As un-Mac-like as Word 6 was, it was far more Mac-like then than Google Docs running inside a Chrome tab is today. Google Docs on Chrome is an un-Mac-like word processor running inside an ever-more-un-Mac-like web browser. What the Mac market flatly rejected as un-Mac-like in 1996 was better than what the Mac market tolerates, seemingly happily, today. Software no longer needs to be Mac-like to succeed on the Mac today. That’s a tragedy.

Unlike Gruber I am not a Mac person but even on Windows I love the performance and integration of native applications that look right, feel right, and take full advantage of the platform.

As a developer I also prefer C# to JavaScript but that is perhaps more incidental – though it shows how far-sighted C# inventor Anders Hejlsberg was when he shifted to work on TypeScript, another super popular open source project from Microsoft.

SQLite with .NET: excellent but some oddities

I have been porting a C# application which uses an MDB database (the old Access/JET format) to one that uses SQLite. The process has been relatively smooth, but I encountered a few oddities.

One is puzzling and is described by another user here. If you have a column that normally stores string values, but insert a string that happens to be numeric such as “12345”, then you get an invalid cast exception from the GetString method of the SQLite DataReader. The odd thing is that the GetFieldType method correctly returns String. You can overcome this by using GetValue and casting the result to a string, or calling GetString() on the result as in dr.GetValue().ToString().

Another strange one is date comparisons. In my case the application only stores dates, not times; but SQLite using the .NET provider stores the values as DateTime strings. The SQLite query engine returns false if you test whether “yyyy-mm-dd 00:00:00” is equal to “yyy-mm-dd”. The solution is to use the date function: date(datefield) = date(datevalue) works as you would expect. Alternatively you can test for a value between two dates, such as more than yesterday and less than tomorrow.

Performance is excellent, with SQLite . Unit tests of various parts of the application that make use of the database showed speed-ups of between 2 and 3 times faster then JET on average; one was 8 times faster. Note though that you must use transactions with SQLite (or disable synchronous operation) for bulk updates, otherwise database writes are very slow. The reason is that SQLite wraps every INSERT or UPDATE in a transaction by default. So you get the effect described here:

Actually, SQLite will easily do 50,000 or more INSERT statements per second on an average desktop computer. But it will only do a few dozen transactions per second. Transaction speed is limited by the rotational speed of your disk drive. A transaction normally requires two complete rotations of the disk platter, which on a 7200RPM disk drive limits you to about 60 transactions per second.

Without a transaction, a unit test that does a bulk insert, for example, took 3 minutes, versus 6 seconds for JET. Refactoring into several transactions reduced the SQLite time to 3 seconds, while JET went down to 5 seconds.

Should you convert your Visual Basic .NET project to C#? Why and why not…

When Microsoft first started talking about Roslyn, the .NET compiler platform, one of the features described was the ability to take some Visual Basic code and “paste as C#”, or vice versa.

Some years later, I wondered how easy it is to convert a VB project to C# using Roslyn. The SharpDevelop team has a nice tool for this, CodeConverter, which promises to “Convert code from C# to VB.NET and vice versa using Roslyn”. You can also find this on the Visual Studio marketplace. I installed it to try out.

image

Why would you do this though? There are several reasons, the foremost of which is cross-platform support. The Xamarin framework can use VB to some extent, but it is primarily a C# framework. .NET Core was developed first for C#. Microsoft has stated that “with regard to the cloud and mobile, development beyond Visual Studio on Windows and for non-Windows platforms, and bleeding edge technologies we are leading with C#.”

Note though that Visual Basic is still under active development and history suggests that your Windows VB.NET project will continue running almost forever (in IT terms that is). Even Visual Basic 6.0 applications still run, though you might find it convenient to keep an old version of Windows running for the IDE.

Still, if converting a project is just a right-click in Visual Studio, you might as well do it, right?

image

I tried it, on a moderately-size VB DLL project. Based on my experience, I advise caution – though acknowledging that the converter does an amazing job, and is free and open source. There were thousands of errors which will take several days of effort to fix, and the generated code is not as elegant as code written for C#. In fact, I was surprised at how many things went wrong. Here are some of the issues:

The tool makes use of the Microsoft.VisualBasic namespace to simplify the conversion. This namespace provides handy VB features like DateDiff, which calculates the difference between two dates. The generated project failed to set a reference to this assembly, generating lots of errors about unknown objects called Information, Strings and so on. This is quick to fix. Less good is that statements using this assembly tend to be more convoluted, making maintenance harder. You can often simplify the code and remove the reference; but of course you might introduce a bug with careless typing. It is probably a good idea to remove this dependency, but it is not a problem if you want the quickest possible port.

Moving from a case-insensitive language to a case-sensitive language is a problem. Visual Studio does a good job of making your VB code mostly consistent with regard to case, but that is not a fix. The converter was unable to fix case-sensitivity issues, and introduced some of its own (Imports System.Text became using System.text and threw an error). There were problems with inheritance, and even subtle bugs. Consider the following, admittedly ugly and contrived, code:

image

Here, the VB coder has used different case for a parameter and for referencing the parameter in the body of the method. Unfortunately another variable with the different case is also accessible. The VB code and the converted C# code both compile but return different results. Incidentally, the VB editor will work very hard to prevent you writing this code! However it does illustrate the kind of thing that can go wrong and similar issues can arise in less contrived cases.

C# is more strict than VB which causes errors in conversion. In most cases this is not a bad thing, but can cause headaches. For example, VB will let you pass object members ByRef but C# will not. In fact, VB will let you pass anything ByRef, even literal values, which is a puzzle! So this compiles and runs:

image

Another example is that in VB you can use an existing variable as the iteration variable, but in C# foreach you cannot.

Collections often go wrong. In VB you use an Item property to access the members of a collection like a DataReader. In C# this is omitted, but the converter does not pick this up.

Overloading sometimes goes wrong. The converter does not always successfully convert overloaded methods. Sometimes parameters get stripped away and a spurious new modifier is added.

Bitwise operators are not correctly converted.

VB allows indexed properties and properties with parameters. C# does not. The converter simply strips out the parameters so you need to fix this by hand. See https://stackoverflow.com/questions/2806894/why-c-sharp-doesnt-implement-indexed-properties if the language choices interest you.

There is more, but the above gives some idea about why this kind of conversion may not be straightforward.

It is probably true that the higher the standard of coding in the original project, the more straightforward the conversion is likely to be, the caveat being that more advanced language features are perhaps more likely to go wrong.

Null strings behave differently

Another oddity is that VB treats a String set to null (Nothing) as equivalent to an empty string:

Dim s As String = Nothing

If (s = String.Empty) Then ‘TRUE in VB
     MsgBox(“TRUE!”)
End If

C# does not:

String s = null;

   if (s == String.Empty) //FALSE in C#
    {
        //won’t run
    }

Same code, different result, which can have unfortunate consequences.

Worth it?

So is it worth it? It depends on the rationale. If you do not need cross-platform, it is doubtful. The VB code will continue to work fine, and you can always add C# projects to a VB solution if you want to write most new code in C#.

If you do need to move outside Windows though, conversion is worthwhile, and automated conversion will save you a ton of manual work even if you have to fix up some errors.

There are two things to bear in mind though.

First, have lots of unit tests. Strange things can happen when you port from one language to another. Porting a project well covered by tests is much safer.

Second, be prepared for lots of refactoring after the conversion. Aim to get rid of the Microsoft.VisualBasic dependency, and use the stricter standards of C# as an opportunity to improve the code.

Gartner on Mobile App Development Platforms: Kony, Mendix, Microsoft, Oracle and Outsystems the winners

Gartner has published a paper and Magic Quadrant on Mobile App Development Platforms (MDAPs), which you can read for free thanks to Progress, pleased to be named as a “Visionary”, and probably from other sources.

According to Gartner, an MDAP has three key characteristics:

  • Cross-platform front-end development tools
  • Back-end services that can be used by diverse clients, not just the vendor’s proprietary tools.
  • Flexibility to support public and internal deployments

Five vendors ranked in the sought-after “Leaders” category. These are:

  • Kony, which offers Kony Visualizer for building clients, Kony Fabric for back-end services, and Kony Nitro Engine, a kind of cross-platform runtime based on Apache Cordova .
  • Mendix, which has visual development and modeling tools and multi-cloud, containerised deployment of back-end services
  • Microsoft, which has Xamarin cross-platform development, Azure cloud services, and PowerApps for low-code development
  • Oracle, which has Oracle Mobile Cloud Enterprise including JavaScript Extension Toolkit and deployment via Apache Cordova
  • Outsystems, a low-code platform which has the Silk UI Framework and a visual modeling language, and hybrid deployment via Apache Cordova

Of course there are plenty of other vendors covered in the report. Further, because this is about end-to-end platforms, some strong cross-platform development tools do not feature at all.

A few observations. One is the prominence of Apache Cordova in these platforms. Personally I have lost enthusiasm for Cordova, now that there are several other options (such as Xamarin or Flutter) for building native code apps, which I feel deliver a better user experience, other things being equal (which they never are).

With regard to Microsoft, Gartner notes the disconnect between PowerApps and Xamarin, different approaches to application development which have little in common other than that both can be used with Azure back-end services.

image
Microsoft PowerApps

I found the report helpful for its insight into which MDAP vendors are successfully pitching their platform to enterprise customers. What it lacks is much sense of which platforms offer the best developer experience, or the best technical capability when it comes to solving those unexpected problems that inevitably crop up in the middle of your development effort and take a disproportionate amount of time and effort to solve.

RemObjects Elements: mix and match languages and platforms as you like

The world of software development has changed profoundly in the last decade or so. Once it was a matter of mainly desktop Windows development for the client, mainly Java for server-based applications with web or Windows clients. Then came mobile and cloud – the iPhone SDK was released in March 2008, kicking off a new wave of mobile applications, while Amazon EC2 (Elastic Compute Cloud) came out of beta in October 2008. Microsoft tussled within itself about what to do with Windows Mobile and ended up ceding the entire market to Android and iOS.

The consequence of these changes is that business developers who once happily developed Windows desktop applications have had to diversify, as their customers demand applications for mobile and web as well. The PC market has not gone away, so there has been growing interest in both cross-platform development and in how to port Windows code to other platforms.

Embarcadero took Delphi, a favourite development tool based on an Object Pascal compiler, down a cross-platform path but not to the satisfaction of all Delphi developers, some of whom looked for other ways to transition to the new world.

Founded in 2002, RemObjects had a project called Chrome, which compiled Delphi’s Object Pascal to .NET executables. This product was later rebranded Oxygene. For a while Embarcadero bundled a version of this with Delphi, calling it Prism, after abandoning its own .NET compilation tools.

The partnership with Embarcadero ended, but RemObjects pressed on, adding language features to its flavour of Object Pascal and adding support for Mac OS X, iPhone and Java.

In February 2015 the company was an early adopter of Apple’s Swift language, introducing a Swift compiler called Silver that targets Android, .NET and native Mac OS X executables.

The company now offers a remarkable set of products for developers who want to target new platforms but in a familiar language:

  • Oxygene: Object Pascal
  • Silver: Swift 3 (and most of Swift 4)
  • Hydrogene: C# 7
  • Iodine: Java 8

Each language can import APIs from the others, and compile to all the platforms – well, there are exceptions, but this is the general approach.

More precisely, RemObjects defines four target platforms:

  • Echoes: .NET and .NET Core including ASP.NET and Mono
  • Cooper: Java and Android
  • Toffee: Mac, iOS, tvOS
  • Island: CPU native and WebAssembly

So if you fancy writing a WPF (Windows Presentation Foundation) application in Java, you can:

image

As you may spot from the above screenshot, the RemObjects tools use Visual Studio as the IDE. This is a limitation for Mac developers, so the company also developed a Mac IDE called Fire, and now a Windows IDE called Water (in preview) for those who dislike the Visual Studio dependency.

image

Important to note: RemObjects does not address the problem of cross-platform user interfaces. In this respect it is similar to the approach taken by Xamarin before that company came up with the idea of Xamarin Forms. So this is about sharing non-visual code and libraries, not cross-platform GUI (Graphical User Interface). If you are targeting Cocoa, you can use Apple’s Interface Builder to design your user interface, for example.

Of course WebAssembly and HTML is an interesting option in this respect.

A notable absentee from the list of RemObjects targets is UWP (Universal Windows Platform), a shame given the importance Microsoft still attaches to this.

RemObjects is mainly focused  on languages and compilers rather than libraries and frameworks. The idea is that you use the existing libraries and frameworks that are native to the platform you are targeting. This is a smart approach for a small company that does not wish to reinvent the wheel.

That said, there is a separate product called Data Abstract which is a multi-tier database framework.

These are interesting products, but as a journalist I have struggled to give them much coverage, because of their specialist nature and also the demands on my time as someone who prefers to try things out rather than simply relay news from press releases. I also appreciate that the above information is sketchy and encourage you to check out the website if these tools pique your interest.

Embarcadero launches free Community Edition of Delphi and C++Builder for mainly non-commercial use

A new Community Edition of Delphi and C++Builder, visual development tools for Windows, Mac, Android and iOS, has been released by Embarcadero.

image

The tools are licensed for non-commercial use or for commercial use (for up to 5 developers) where revenue is less than $5000 per year. It is not totally clear to me, but I believe this means the total revenue (or for non-profits, donations) of the individual or organisation, not just the revenue generated by Community Edition applications. From the EULA:

The Community Edition license applies solely if Licensee cumulative annual revenue (of the for-profit organization, the government entity or the individual developer) or any donations (of the non-profit organization) does not exceed USD $5,000.00 (or the equivalent in other currencies) (the “Threshold”). If Licensee is an individual developer, the revenue of all contract work performed by developer in one calendar year may not exceed the Threshold (whether or not the Community Edition is used for all projects).

Otherwise, the Community Editions are broadly similar to the Professional Editions of these tools. Note that even the Professional Edition lacks database drivers other than for local or embedded databases so this is a key differentiator in favour of the Architect or Enterprise editions.

An annoyance is that you cannot install both Delphi and C++ Builder Community Editions on the same PC. For this you need RAD Studio which has no Community Edition.

Delphi and C++ Builder are amazing tools for Windows desktop development, with a compiler that generates fast native code. For cross-platform there is more competition, not least from Microsoft’s Xamarin tools, but the ability to share code across multiple platforms has a powerful attraction.

Get Delphi Community Edition here and C++Builder Community Edition here.

Using the Xamarin WebView for programmatic display of HTML content

Xamarin Forms is a key framework for C# and .NET developers since it lets you target Android, iOS and to some extent Windows (UWP and therefore Windows 10 only) with maximum code reuse. I have a longstanding interest in embedded web browser controls and was glad to see that Xamarin Forms supports a capable WebView control. The WebView wraps Chrome on Android, Safari on iOS, and Edge on UWP.

I did a quick hands-on. In this example (running in the Android emulator on Hyper-V, of course), the HTML is generated programmatically and the CSS loaded from local storage. I also added some script to show the User Agent string that identifies the browser.

image

There are a few things needed to make this work. Some XAML to put the WebView on a page. Then to load content into the WebView you need an HTMLWebViewSource object. If you are loading external files, you must set the BaseUrl property of this object as well as the HTML itself. The BaseUrl tells the control where to look for files that have a relative address. This varies according to the target platform, so you use the Xamarin Forms Dependency Service to set it correctly for each platform.

In Visual Studio, you place the files you want to load in the appropriate folder for each platform. For Android, this is the Assets folder.

That is about all there is to it. As you can see from the above screenshot, I wrote very little code.

The WebView control can also display PDF documents. Finally, there is an EvaluateJavaScriptAsync method that lets you call JavaScript in a WebView and read the results from C#.

This JavaScript bridge is a workaround for the most obvious missing feature, that you cannot directly read the HTML content from the WebView. If this is a full programmatic solution and you generate all the HTML yourself, you can add JavaScript to do what you want. If the user is allowed to navigate anywhere on the web, you cannot easily grab the HTML; but this could be a good thing, in case the user entered a password or is viewing confidential data. You can grab the destination URL from the Navigating event and read it separately if necessary. But the intent of the control is to let you create rich applications that take advantage of the browser’s ability to render content, not to invade a user’s privacy by tracking their web browsing.

Instant applications considered harmful?

Adrian Colyer, formerly of SpringSource, VMWare, and Pivotal, is running an excellent blog where he looks at recent technical papers. A few days ago he covered The Rise of the Citizen Developer – assessing the security impact of online app generators. This was about online app generators for Android, things like Andromo which let you create an app with a few clicks. Of course the scope of such apps is rather limited, but they have appeal as a quick way to get something into the Play Store that will promote your brand, broadcast your blog, convert your website into an app, or help customers find your office.

It turns out that there are a few problems with these app generators. Andromo is one of the better ones. Some of them just download a big generic application with a configuration file that customises it to your requirements. Often this configuration is loaded from the internet, in some cases over HTTP with no encryption. API keys used for interaction with other services such as Twitter and Google can easily leak. They do not conform to Android security best practices and request more permissions that are needed.

Low code or no-code applications are not confined to Android applications. Appian promises “enterprise-grade” apps via its platform.  Microsoft PowerApps claims to “solve business problems with intuitive visual tools that don’t require code.” It is an idea that will not go away: an easy to use visual environment that will enable any business person to build productive applications.

Some are better than others; but there are inherent problems with all these kinds of tools. Three big issues come to mind:

  1. Bloat. You only require a subset of what the application generator can do, but by trying to be universal there is a mass of code that comes along with it, which you do not require but someone else may. This inevitably impacts performance, and not in a good way.
  2. Brick walls. Everything is going well until you require some feature that the platform does not support. What now? Often the only solution is to trash it and start again with a more flexible tool.
  3. Black box. You app mostly works but for some reason in certain cases it gives the wrong result. Lack of visibility into what it happening behind the scenes makes problems like this hard to fix.

It is possible for an ideal tool to overcome these issues. Such a tool generates human-understandable code and lets you go beyond the limitations of the generator by exporting and editing the project in a full programming environment. Most of the tools I have seen do not allow this; and even if they do, it is still hard for the generator to avoid generating a ton of code that you do not really need.

The more I have seen of different kinds of custom applications, the more I appreciate projects with nicely commented textual code that you can trace through and understand.

The possibility of near-instant applications has huge appeal, but beware the hidden costs.