First iOS App
March 22, 2022 22:05
Machine learning Electron Microscope Image processing iOS app!
App is published!
A Machine Learning App for all Platforms
Starting Summer 2021, with the support of a grant from UCI's Summer Undergraduate Research Proposal, I began developing MatLab and iOS versions for the processing of Atom Resolution images. The machine learning model was trained in UCI Professor Huolin Xin's group; you can read the original publication
. The machine learning originates in python, but I have finished developing the iOS version March 2022, published July (I dealt with some publishing problems with Apple). The app will allow exploration of machine learning in a user-friendly way. A huge benefit is to show the non-coding public a neural network in action!
I dont know why it says "designed for iPad" on the app store. XD
The app is built using SwiftUI, apple's newest User Interface API.
SwiftUI uses a declarative programming style
Model View ViewModel or MVVM
. The MVVM style of programming has the benefit of being easier to read and debug. SwiftUI also makes it easy to develop for more apple platforms once one version is complete, easier than it was using UIKit.
The machine learning model is powered by
Apple's (also cutting edge) machine learning API. CoreML is so new, it is still impossible as of March 2022 to design your own UNet architecture in
. Instead you must use a python tool (
) to convert the models from Pytorch to CoreML.
This was a breakthrough in my project, however some boilerplate code was needed. For this I designed an image processing pipeline which converts PNGs to CGImage, CGImage data to MLMultiArray for model input, and MLMultiArray back to CGImage for model output.
This is all blazing fast thanks to low level frameworks.
With the pipeline working, I implemented persistent data stores (Core Data) to hold images permanently in the app.
For a finished version, import and export capability allow the user to share images.
This app is a great example of implementing machine learning for everyone. This project was one of my most exciting efforts because I felt like all the effort self learning coding in Swift payed off.
Thank you to Professor Xin and UCI for supporting me in producing this app.
August 09, 2022 18:51
Add a Comment:
sign in to comment.