Native Mac APIs for Go(github.com)
github.com
Native Mac APIs for Go
https://github.com/progrium/macdriver
71 comments
[deleted]
I don't see an @autoreleasepool anywhere in the source, which I believe means calling some AppKit APIs from Go-managed threads will silently leak memory (due to memory allocated internally by the APIs that will never get released).
In my own projects, I had to wrap calls from foreign threads into AppKit APIs with an @autoreleasepool {} block or my app would leak memory. [1]
[1] https://developer.apple.com/library/archive/documentation/Co...
In my own projects, I had to wrap calls from foreign threads into AppKit APIs with an @autoreleasepool {} block or my app would leak memory. [1]
[1] https://developer.apple.com/library/archive/documentation/Co...
I thought I wrapped NSAutoreleasePool but maybe I just used it directly dynamically. If something is not wrapped in the source that doesn't mean you can't use it. objc.Get("NSAutoreleasePool").Alloc().Init()
Unfortunately I can afford some leaks at the moment, so if that's critical to anybody else and I'm doing something wrong just submit a PR
Unfortunately I can afford some leaks at the moment, so if that's critical to anybody else and I'm doing something wrong just submit a PR
Ah, I don't just mean the class, I mean you need to have an autoreleasepool block active before calling into AppKit otherwise you can leak memory on every call. It doesn't look like you're using pools yet, or documenting that users of your library should use them.
See here: https://developer.apple.com/library/archive/documentation/Co...
> Cocoa always expects code to be executed within an autorelease pool block, otherwise autoreleased objects do not get released and your application leaks memory
See here: https://developer.apple.com/library/archive/documentation/Co...
> Cocoa always expects code to be executed within an autorelease pool block, otherwise autoreleased objects do not get released and your application leaks memory
I guess I will look into this as that really sounds like syntactic sugar for something more basic. Like using the class.
I have a hard time keeping up with their changes but you might be right: https://developer.apple.com/documentation/foundation/nsautor...
Oddly it says you cannot use them directly, but later implies maybe they are just less efficient. It would be nice if somebody made an issue for this.
I have a hard time keeping up with their changes but you might be right: https://developer.apple.com/documentation/foundation/nsautor...
Oddly it says you cannot use them directly, but later implies maybe they are just less efficient. It would be nice if somebody made an issue for this.
The page you linked is not actually ambiguous, though perhaps a bit tricky to read. It says:
1. If you're compiling Objective C in ARC mode, you can't use NSAutoreleasePool directly, and must instead use @autoreleasepool.
2. In manual reference counting mode you can use either NSAutoreleasePool or @autoreleasepool, but the latter has lower overhead. (This may matter if e.g. you're draining the autorelease pool on every iteration of a loop to reduce memory spikes.)
Under the hood -- at least on the version I disassembled -- NSAutoreleasePool's -init and -release methods wrap the CoreFoundation CFAutoreleasePoolPush and CFAutoreleasePoolPop functions, which in turn call the runtime's objc_autoreleasePoolPush and objc_autoreleasePoolPop functions, which are the things that @autoreleasepool will cause the compiler to emit directly.
1. If you're compiling Objective C in ARC mode, you can't use NSAutoreleasePool directly, and must instead use @autoreleasepool.
2. In manual reference counting mode you can use either NSAutoreleasePool or @autoreleasepool, but the latter has lower overhead. (This may matter if e.g. you're draining the autorelease pool on every iteration of a loop to reduce memory spikes.)
Under the hood -- at least on the version I disassembled -- NSAutoreleasePool's -init and -release methods wrap the CoreFoundation CFAutoreleasePoolPush and CFAutoreleasePoolPop functions, which in turn call the runtime's objc_autoreleasePoolPush and objc_autoreleasePoolPop functions, which are the things that @autoreleasepool will cause the compiler to emit directly.
This answer says the block is more efficient than manually managing NSAutoreleasePool objects: https://stackoverflow.com/a/12448176
This answer looks like a better overview of what the runtime is doing: https://stackoverflow.com/a/21010442
The @autoreleasepool block seems equivalent to this:
This answer looks like a better overview of what the runtime is doing: https://stackoverflow.com/a/21010442
The @autoreleasepool block seems equivalent to this:
ctx = _objc_autoreleasePoolPush()
defer _objc_autoreleasePoolPop(ctx)
The implementation of your "sugar" can be shortened:
https://play.golang.org/p/8v4EL2B_c_t
https://play.golang.org/p/8v4EL2B_c_t
Awesome, can you throw into an issue?
wow, thanks!
Keep in mind that the reason the `@autorelease` syntax is faster is primarily due to ARC optimizations (which don't apply here, since you're not using ARC).
Calling the `_objc_autoreleasePoolXX` functions are still likely to be faster than the NSAutoreleasePool objects, but only because you're avoiding the Objective-C message sends.
Calling the `_objc_autoreleasePoolXX` functions are still likely to be faster than the NSAutoreleasePool objects, but only because you're avoiding the Objective-C message sends.
Most of Jeff Lindsay's stuff is worth checking out. He also started dokku and coined the term webhook.
We use Dokku on the regular for a number of small services at work. It was a little strange being new to it, but it's been great. Kudos to him.
I know it's kind of going against the idea of Dokku, but can it be made to scale across servers? I love the simplicity of Heroku but the cost can be a barrier for some use cases.
But k8s is too complicated. Anything like Dokku that scales horizontally?
The scheduler in Dokku is pluggable. You can use Kubernetes or Nomad: http://dokku.viewdocs.io/dokku/advanced-usage/schedulers/alt...
How did I not know that? How "prod" ready is Dokku if we just ran it on top of k8s?
What's another layer of abstraction on top of _Kubernetes_. #yolo
I haven't actually used k8s-backed Dokku, but I imagine that it would be worth at least experimenting with if you're already committed to running stuff on k8s.
I haven't actually used k8s-backed Dokku, but I imagine that it would be worth at least experimenting with if you're already committed to running stuff on k8s.
If you're looking for a cost-effective alternative to Heroku, might I suggest signing up for my product https://primcloud.com. We're pushing towards public availability by end of February early March at the absolute latest.
I'm a former lover of Heroku and maintainer of Dokku.
I'm a former lover of Heroku and maintainer of Dokku.
If I were avoiding heroku because of cost and had a need to scale horizontally, I'd use Fargate with spot instances.
Manage it with CDK or terraform. As long as you have enough containers running, spot ends up being a non issue.
Manage it with CDK or terraform. As long as you have enough containers running, spot ends up being a non issue.
It's even simpler now, lambda supports running docker images as of a couple months ago: https://aws.amazon.com/blogs/aws/new-for-aws-lambda-containe...
I also recommend this path, but I found it really hard to write all of the required Terraform for the VPC, security groups, load balancers, and Fargate configuration.
So I put together an open-source Terraform super-module to automatically set all of that up in a few lines of code.
https://provose.com/
So I put together an open-source Terraform super-module to automatically set all of that up in a few lines of code.
https://provose.com/
CDK has `ecs-patterns` https://github.com/aws/aws-cdk/tree/master/packages/%40aws-c...
I'm a pretty big fan of CDK if you are willing to make cloudformation and AWS your lingua franca. I use it for most of my personal projects, but professionally bias towards terraform.
I'm a pretty big fan of CDK if you are willing to make cloudformation and AWS your lingua franca. I use it for most of my personal projects, but professionally bias towards terraform.
My application is actually pretty bandwidth heavy, which removes AWS and other major cloud operators from the equation.
But I’ll keep it in mind for non bandwidth intensive workloads.
This is the same concern I had. I settled on Caprover (https://caprover.com/), seems to exactly fit the bill.
shameless plug... you might be interested in https://apppack.io. I built it for very similar reasons.
flynn.io was the product I used found when I wanted a little better scalability than Dokku. This was a few years ago when I tried it, so unsure of the current landscape.
I had tried flynn in the past (a few years ago just as you) but couldn't get it to work, but that was with Meteor which has very specific deployment requirements. I'll try out flynn when I need to scale beyond Dokku.
For my hobby servers I have tried dokku, Flynn and countless other similar solutions. Settled with caprover[1] and very happy with it
[1] https://caprover.com
[1] https://caprover.com
Look at a hosted knative solution, like Google Cloud run or one of many others: https://knative.dev/docs/knative-offerings/ You don't have to know or care about anything Kubernetes with it, but you get ease of just throwing containers at something and having it run them for web-scenarios. They scale up and down to zero so you aren't paying a cost when nothing is happening. And when/if you need it, you have the full power of a kubernetes cluster at your fingertips too.
AWS lambda can run plain old docker containers now too. Check out something like the serverless framework to make it easy to define a bunch of web services or APIs and deploy to lambda, knative, etc.
AWS lambda can run plain old docker containers now too. Check out something like the serverless framework to make it easy to define a bunch of web services or APIs and deploy to lambda, knative, etc.
also wrote registrator which helped pioneer the technique of service discovery we see in so many cloud tools and platforms today.
This looks really cool! Is there any overhead with using these bindings as opposed to using Swift and calling the APIs directly?
> Is there any overhead with using these bindings
Not the author, but yes. Go calling C is expensive, and that's precisely how this works. Whether or not that matters for your use case is another topic.
Not the author, but yes. Go calling C is expensive, and that's precisely how this works. Whether or not that matters for your use case is another topic.
There's a bit of a write-up here:
https://www.cockroachlabs.com/blog/the-cost-and-complexity-o...
In their benchmark, calling a `func() {}` in Go vs a `void foo() {}` in C (via CGo) is almost 100x faster.
https://www.cockroachlabs.com/blog/the-cost-and-complexity-o...
In their benchmark, calling a `func() {}` in Go vs a `void foo() {}` in C (via CGo) is almost 100x faster.
$ go test -bench . -gcflags '-l' # disable inlining for fairness
BenchmarkCGO-8 10000000 171 ns/op
BenchmarkGo-8 2000000000 1.83 ns/op
But if the called function takes more than 200ns to execute, the overhead is less than 50%. Which probably is true for most relevant functions in an API
Go calling C is expensive compared to the normal function call overhead, but if the called function does significant work, should not introduce too much overhead.
> Go calling C is expensive
Is this true even if only primitives are passed and returned? If so, why? For comparison, Java is fast at that these days, if I understand correctly.
(Interesting related reading regarding Java: https://web.archive.org/web/20160304055443/http://nerds-cent... )
Is this true even if only primitives are passed and returned? If so, why? For comparison, Java is fast at that these days, if I understand correctly.
(Interesting related reading regarding Java: https://web.archive.org/web/20160304055443/http://nerds-cent... )
This is a good question that should be added in the readme. Would love an issue for it to also collect data
It's a little sad that after 50 years since Microsoft and Apple were started, there still is no easy way to write a natively looking cross-platform GUI app, even with a language like Golang.
Recently I looked at couple of options of writing some Golang to make a simple GUI app that would look natively on Windows/MacOS (text area, several checkboxes/buttons), and there's no much there to be honest.
Recently I looked at couple of options of writing some Golang to make a simple GUI app that would look natively on Windows/MacOS (text area, several checkboxes/buttons), and there's no much there to be honest.
Sure there is: Write a separate UI for each platform.
Seriously, this is the only option if you care not just about looks but also about feel, because no cross-platform toolkit gets things like keyboard shortcuts, scrolling inertia, menu mnemonics, etc. correct on all platforms they target. Not to mention platform accessibility features.
Surely most of the work in your app is platform-independent, right? Plumbing data into platform-specific UIs should not be a hard endeavor.
Seriously, this is the only option if you care not just about looks but also about feel, because no cross-platform toolkit gets things like keyboard shortcuts, scrolling inertia, menu mnemonics, etc. correct on all platforms they target. Not to mention platform accessibility features.
Surely most of the work in your app is platform-independent, right? Plumbing data into platform-specific UIs should not be a hard endeavor.
Fortunately you can go four levels of virtualization in and do it pretty easily
Users won’t care, there’s no award
Users won’t care, there’s no award
Unironically, your best option is GTK. I've found that it generates consistent looking UIs for Linux and Windows (my main build targets) without totally ruling out MacOS
Sure there is, my C++ code has worked just fine all these years.
I think Microsoft wants to address that in .net 6
D has native support for calling Objective-C. No need to integrate with the Objective-C runtime directly. Everything is written using familiar D syntax. No extra overhead compared to Objective-C. The generated D code is the same as the Objective-C compiler would generate.
https://dlang.org/spec/objc_interface.html
https://dlang.org/spec/objc_interface.html
Although it's a bit out of the etiquete to post tangents like this, I find it really cool that D has such native support and will be reading more about it because you shared. So thanks.
I don’t think it’s out of line. Oftentimes comments like this do come with jabs at the language being discussed, but this one was pleasantly free of snark. It makes me more curious about D.
This is great, I've done a few one-off go wrappers of objective C libraries [1][2] for an OSX menuapp framework I built [3].
This seems like a much more general and useful solution, excited to switch some things over to it!
[1] https://github.com/caseymrm/go-pmset
[2] https://github.com/caseymrm/go-smc
[3] https://github.com/caseymrm/menuet
This seems like a much more general and useful solution, excited to switch some things over to it!
[1] https://github.com/caseymrm/go-pmset
[2] https://github.com/caseymrm/go-smc
[3] https://github.com/caseymrm/menuet
Great work, OP.
However, I think it's problematic that modern languages (Go, Rust) don't provide these kinds of bindings out of the box. I honestly think it's probably starting to be "in-scope" for standard libraries. I'm working on a side-project now which needs a desktop app, and I'm actually using Fyne[1] because it makes cross-platform development semi-easy (even though it doesn't look native, nor particularly great). The alternative is using either Electron or using something like this for OSX, using something else for Windows, and using something else for Linux.
Window/widget/notification/taskbar APIs are stable for all major operating systems, and it seems like we keep reinventing the wheel here.
[1] https://fyne.io/
However, I think it's problematic that modern languages (Go, Rust) don't provide these kinds of bindings out of the box. I honestly think it's probably starting to be "in-scope" for standard libraries. I'm working on a side-project now which needs a desktop app, and I'm actually using Fyne[1] because it makes cross-platform development semi-easy (even though it doesn't look native, nor particularly great). The alternative is using either Electron or using something like this for OSX, using something else for Windows, and using something else for Linux.
Window/widget/notification/taskbar APIs are stable for all major operating systems, and it seems like we keep reinventing the wheel here.
[1] https://fyne.io/
Do you mean we should have OS-specific methods in languages' standard libraries? Or that we should add cross-platform frameworks to languages' standard libraries?
I am freaked out by the line, "Retain and Release methods for working with Objective-C garbage collection".
Do they mean the abandoned GC system Apple tried, or the obsolete manual pool-based system Apple used to use?
Either way, I want ARC instead.
Do they mean the abandoned GC system Apple tried, or the obsolete manual pool-based system Apple used to use?
Either way, I want ARC instead.
Retain/Release methods would be for manual reference counting, as is traditional in Obj-C/Cocoa.
ARC is a compiler trick in the clang objective-c and swift compilers-- essentially injecting the appropriate retain/release calls for you. You won't get that for free in Go (because, again, it's a compiler feature-- compiled ObjC/Swift code is still calling retain/release just like you would manually), but I'd expect some integration with Go's memory management system so you're not having to make those calls manually. (It looks like that is not here in this particular release, though-- they have retain/release calls in some of their examples.)
Objective-C garbage collection is not a thing any more, and hasn't been for quite some time. Deprecated back in macOS 10.8, and outright removed in more recent versions of the runtime (it was never available on iOS, and was taken out sometime around macOS 10.12 on desktop).
ARC is a compiler trick in the clang objective-c and swift compilers-- essentially injecting the appropriate retain/release calls for you. You won't get that for free in Go (because, again, it's a compiler feature-- compiled ObjC/Swift code is still calling retain/release just like you would manually), but I'd expect some integration with Go's memory management system so you're not having to make those calls manually. (It looks like that is not here in this particular release, though-- they have retain/release calls in some of their examples.)
Objective-C garbage collection is not a thing any more, and hasn't been for quite some time. Deprecated back in macOS 10.8, and outright removed in more recent versions of the runtime (it was never available on iOS, and was taken out sometime around macOS 10.12 on desktop).
age old problem, i should definitely warn people about memory management implications. what would you put in the readme for this?
Do the BridgeSupport files annotate whether you own returned objects and need to release them? Sprinkling in runtime.SetFinalizer calls based on ownership would be slightly nicer than exposing release/retain.
Manual Reference Counting (aka Manual Retain Release) is obsolete in the sense that there's no reason for most people to use it, but it is still completely supported, and Apple themselves still have huge MRC codebases, including AppKit itself. Automatic Reference Counting (ARC) uses the same underlying reference counting mechanism, it just inserts the retain/release calls for you at compile time. (There's slightly more to it than that, especially around ARC's ability to optimize, but it's not too far off.)
The author has already clarified that Objective-C "garbage collection" wasn't really what he meant, but the distinction between ARC and MRC only makes sense in the context of the ObjC compiler itself. From the "outside", esp. for compiled binaries (like the system frameworks), ObjC classes always just use reference counting.
The author has already clarified that Objective-C "garbage collection" wasn't really what he meant, but the distinction between ARC and MRC only makes sense in the context of the ObjC compiler itself. From the "outside", esp. for compiled binaries (like the system frameworks), ObjC classes always just use reference counting.
Last I tried to bind to ObjC APIs through C you needed to use NSAutoReleasePool at some point, and to interact with the GC as the APIs expect.
There was no option to use anything else. Has that changed?
There was no option to use anything else. Has that changed?
ARC is built on top of the retain/release pool system, but it automates everything from the programmer's point of view so you almost never have to explicitly interact with it.
sorry I meant memory management not garbage collection. again they're just convenience methods wrapping those exact methods on NSObject
Bindings for Apple's APIs for other languages come out every once in a while. For example, PyObjC for Python, RubyCocoa and MacRuby, Apple's own old Java bindings, etc.
Usually these garner early interest, but no significant programs come out using them, and they tend to be abandoned within a few years. I don't mean to discourage the author, but I wonder if he's aware of the history and if he has thoughts on why Go may fare better.
Usually these garner early interest, but no significant programs come out using them, and they tend to be abandoned within a few years. I don't mean to discourage the author, but I wonder if he's aware of the history and if he has thoughts on why Go may fare better.
Not going to do a GH issue as it says this area is very much in flux but it looks like the bridge documentation is outdated.
Otherwise this worked really well and intuitively for a golang dev!
Otherwise this worked really well and intuitively for a golang dev!
We used a lot of PyObjC BITD at Apple to explore frameworks that we couldn't get documentation for. That was simultaneously awesome and so so so broken, from an organizational perspective.
Looks cool. What about Swift support?
That would be a lot more difficult (unless you limited support to the subset of Swift which is shared by Objective-C). Objective-C is an extremely simple extension to C, and has a rich runtime API. So any language which can interface with C can interface with Objective-C by using the runtime interfaces.
I successfully used the other way - calling Golang functions from iOS/Swift using Gomobile. It is working now for the new M1 Apple desktop computers too.
For those interested in a Rust variant, I've been hacking on one for awhile: https://github.com/ryanmcgrath/cacao
I wouldn't say it's yet ready for production use, but so far it's working pretty well for me. Been dogfooding it by building an app I've wanted for a bit - a proper magic-wormhole macOS app.
One of the things I've really wanted to keep to is the delegate pattern, since I think it actually works really well for Rust's model. A fun example I finished yesterday is ListView cell reuse:
https://twitter.com/ryanmcgrath/status/1357097991081844737/p...
Ultimately I view this as one of the last pieces needed for a cross-platform Rust UI framework to actually work.