Code Monkey home page Code Monkey logo

swindler's Introduction

Swindler

A Swift window management library for macOS

Build Status Join the chat at https://gitter.im/tmandry/Swindler

In the past few years, many developers formerly on Linux and Windows have migrated to Mac for their excellent hardware and UNIX-based OS that "just works".

But along the way we gave up something dear to us: control over our desktop environment.

The goal of Swindler is to help us take back that control, and give us the best of both worlds.

What Swindler Does

Writing window managers for macOS is hard. There are a lot of systemic challenges, including limited and poorly-documented APIs. All window managers on macOS must use the C-based accessibility APIs, which are difficult to use and are surprisingly buggy themselves.

As a result, the selection of window managers is pretty limited, and many of the ones out there have annoying bugs, like freezes, race conditions, "phantom windows", and not "seeing" windows that are actually there. The more sophisticated the window manager is, the more it relies on these APIs and the more these bugs start to show up.

Swindler's job is to make it easy to write powerful window managers using a well-documented Swift API and abstraction layer. It addresses the problems of the accessibility API with these features:

Type safety

Swindler's API is fully documented and type-safe thanks to Swift. It's much easier and safer to use than the C-based accessibility APIs. (See the example below.)

In-memory model

Window managers on macOS rely on IPC: you ask an application for a window's position, wait for it to respond, request that it be moved or focused, then wait for the application to comply (or not). Most of the time this works okay, but it works at the mercy of the remote application's event loop, which can lead to long, multi-second delays.

Swindler maintains a model of all applications and window states, so your code knows everything about the windows on the screen. Reads are instantaneous, because all state is cached within your application's process and stays up to date. Swindler is extensively tested to ensure it stays consistent with the system in any situation.

Asynchronous writes and refreshes

If you need to resize a lot of windows simultaneously, for example, you can do so without fear of one unresponsive application holding everything else up. Write requests are dispatched asynchronously and concurrently, and Swindler's promise-based API makes it easy to keep up with the state of operations.

Friendly events

More sophisticated window managers have to observe events on windows, but the observer API is not well documented and often leaves out events you might expect, or delivers them in the wrong order. For example, the following situation is common when a new window pops up:

1. MainWindowChanged on com.google.chrome to <window1>
2. WindowCreated on com.google.chrome: <window1>

See the problem? With Swindler, all events are emitted in the expected order, and missing ones are filled in. Swindler's in-memory state will always be consistent with itself and with the events you receive, avoiding many bugs that are difficult to diagnose.

As a bonus, events caused by your code are marked as such, so you don't respond to them as user actions. This feature alone makes a whole new level of sophistication possible.

Example

The following code assigns all windows on the screen to a grid. Note the simplicity and power of the promise-based API. Requests are dispatched concurrently and in the background, not serially.

Swindler.initialize().then { state -> Void in
    let screen = state.screens.first!

    let allPlacedOnGrid = screen.knownWindows.enumerate().map { index, window in
        let rect = gridRect(screen, index)
        return window.frame.set(rect)
    }

    when(allPlacedOnGrid) { _ in
        print("all done!")
    }
}.catch { error in
    // ...
}

func gridRect(screen: Swindler.Screen, index: Int) -> CGRect {
    let gridSteps = 3
    let position  = CGSize(width: screen.width / gridSteps,
                           height: screen.height / gridSteps)
    let size      = CGPoint(x: gridSize.width * (index % gridSteps),
                            y: gridSize.height * (index / gridSteps))
    return CGRect(origin: position, size: size)
}

Watching for events is simple. Here's how you would implement snap-to-grid:

swindlerState.on { (event: WindowMovedEvent) in
    guard event.external == true else {
        // Ignore events that were caused by us.
        return
    }
    let snapped = closestGridPosition(event.window.frame.value)
    event.window.frame.value = snapped
}

Requesting permission

Your application must request access to the trusted AX API. To do this, simply use this code in your AppDelegate:

func applicationDidFinishLaunching(_ aNotification: Notification) {
    guard AXSwift.checkIsProcessTrusted(prompt: true) else {
        print("Not trusted as an AX process; please authorize and re-launch")
        NSApp.terminate(self)
        return
    }

    // your code here
}

A note on error messages

Many helper or otherwise "special" app components don't respond to the AX requests or respond with an error. As a result, it's expected to see a number of messages like this:

<Debug>: Window <AXUnknown "<AXUIElement 0x610000054eb0> {pid=464}" (pid=464)> has subrole AXUnknown, unwatching
<Debug>: Application invalidated: com.apple.dock
<Debug>: Couldn't initialize window for element <AXUnknown "<AXUIElement 0x610000054eb0> {pid=464}" (pid=464)> () of com.google.Chrome: windowIgnored(<AXUnknown "<AXUIElement 0x610000054eb0> {pid=464}" (pid=464)>)
<Notice>: Could not watch application com.apple.dock (pid=308): invalidObject(AXError.NotificationUnsupported)
<Debug>: Couldn't initialize window for element <AXScrollArea "<AXUIElement 0x61800004ed90> {pid=312}" (pid=312)> (desktop) of com.apple.finder: AXError.NotificationUnsupported

Currently these are logged because it's hard to determine if an app "should" fail (especially on timeouts). As long as things appear to be working, you can ignore them.

Project Status

Swindler is in development and is in alpha. Here is the state of its major features:

  • Asynchronous property system: 100% complete
  • Event system: 100% complete
  • Window API: 90% complete
  • Application API: 90% complete
  • Screen API: 90% complete
  • Spaces API: 0% complete

You can see the entire planned API here.

API Documentation (latest release)

API Documentation (main)

Development

Swindler uses Swift Package Manager.

Building the project

Clone the project, then in your shell run:

$ cd Swindler
$ git submodule init
$ git submodule update

At this point you should be able to build Swindler in Xcode and start on your way!

Using the command line

You can run the example project from the command line.

swift run

Contact

You can chat with us on Gitter.

Follow me on Twitter: @tmandry

Related Projects

Swindler is built on AXSwift.

swindler's People

Contributors

dtweston avatar gitter-badger avatar kcrca avatar shenyj avatar tmandry avatar tombell avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

swindler's Issues

No license specified?

I would love to use some of this code (particularly the NSScreen extensions), but you haven’t specified a license anywhere. Is this intentional, or do you plan to release the code with a BSD or MIT style license?

Upgrade to PromiseKit 6

PromiseKit 6 renames a bunch of things to fix the annoying type annotations needed everywhere. That definitely needs to be passed on to users of Swindler.

Cleanly separate delegates from public classes

Right now the Delegate code has to create and accept "materialized" public classes (State, Application, and Window), which in turn depend on the delegates, for implementing object properties and in order to emit events. An abstraction layer which converts between the two is needed so there's clean separation of concerns.

NSScreen.Screens Coordinates don't align with AXUIElement coordinates for multi-monitor displays

return NSScreen.screens.map{ OSXScreenDelegate(nsScreen: $0) }

I'm doing my own research here, but I think you'll find it relevant based on the code. Right now I have a laptop, with two 1080p monitors. So 3 screens in total. The laptop display is "arranged" roughly as shown below (with the laptop display below the monitor displays, but right aligned with the monitor above):

[1080P monitor][1080p monitor]
  [   laptop  ]

However, with all that being said, different monitor displays don't seem to fix the issue I'm finding (read below).

Using this code:

    var counter = 0
    for screen in NSScreen.screens {
        print("\(counter): \(screen.frame)\t\(screen.visibleFrame)")
        counter += 1
    }

I get the following output:

0: (0.0, 0.0, 1440.0, 900.0)	(0.0, 80.0, 1440.0, 797.0) //laptop display
1: (-480.0, 900.0, 1920.0, 1080.0)	(-480.0, 900.0, 1920.0, 1080.0) // monitor up and to the left from laptop
2: (1440.0, 900.0, 1920.0, 1080.0)	(1440.0, 900.0, 1920.0, 1080.0) // monitor up and to the right from laptop

However, when I put a window on screen 1 or 2, the Accessibility Inspector reports y coordinates that are negative. For example, a full screen window on screen #1 has a Frame of x=-480, y=-1080, w=1920, h=1080.

Do you know why, and how to account for, the fact that the screens through NSScreen.screens reports a different Y value (y = 900 when I think it should be -1080).

This is all on macOS Mojave 10.14.5, on a Macbook Pro 13" 2018 if that's worth anything.

Edit: May have something to do with "quartz" coordinates (see https://stackoverflow.com/questions/22671916/cocoa-getting-the-top-left-logical-point-of-multiple-screens).
Edit 2: Might have a winner https://stackoverflow.com/questions/19884363/in-objective-c-os-x-is-the-global-display-coordinate-space-used-by-quartz-d#answer-19887161

Window level

Is it possible to manipulate a window level using this library? I'm trying to set NSFloatingWindowLevel for an external app.

fatal error: State can't become invalid

A simple call to either:

let primaryScreen: Screen = Swindler.state.screens.first!

or

let windows: [Window] = Swindler.state.knownWindows

in the view controller after the call to super:

override func viewDidLoad() {
    super.viewDidLoad()
}

causes assertionFailure("State can't become invalid") and the app is forced to exit. I tried having different combinations of open windows (Chrome, Sublime, etc.) including none. For some mysterious reason, only when I have Atom open the invalid state assertion doesn't get called and I only get the standard debug/notice log messages in the console.

I tried using a deferred call in case this happens due to a race condition:

override func viewDidLoad() {
    super.viewDidLoad()
    DispatchQueue.main.asyncAfter(deadline: .now() + .milliseconds(5000), execute: {
        let primaryScreen: Screen = Swindler.state.screens.first!
    }
}

without any luck.

My project is a standard Cocoa application and versions info:

OS: Sierra 10.12.6
Xcode: 8.3.3 (8E3004b)
Swift: 3.1

Error & Traceback:

state_invalid_error

fatal error: State can't become invalid: file /Users/stphivos/Projects/Swindler/Swindler/State.swift, line 344
2017-09-21 03:52:19.607833+0300 TestProject[31242:2375701] fatal error: State can't become invalid: file /Users/stphivos/Projects/Swindler/Swindler/State.swift, line 344
Sep 21 03:52:19  TestProject[31242] <Error>: fatal error: State can't become invalid: file /Users/stphivos/Projects/Swindler/Swindler/State.swift, line 344
Current stack trace:
0    libswiftCore.dylib                 0x000000010056a130 swift_reportError + 129
1    libswiftCore.dylib                 0x0000000100586a40 _swift_stdlib_reportFatalErrorInFile + 100
2    libswiftCore.dylib                 0x0000000100377ba0 (_assertionFailure(StaticString, String, file : StaticString, line : UInt, flags : UInt32) -> Never).(closure #1).(closure #1).(closure #1) + 124
3    libswiftCore.dylib                 0x000000010052e360 partial apply for (_assertionFailure(StaticString, String, file : StaticString, line : UInt, flags : UInt32) -> Never).(closure #1).(closure #1).(closure #1) + 93
4    libswiftCore.dylib                 0x0000000100377250 specialized specialized StaticString.withUTF8Buffer<A> ((UnsafeBufferPointer<UInt8>) -> A) -> A + 342
5    libswiftCore.dylib                 0x0000000100530880 partial apply for (_assertionFailure(StaticString, String, file : StaticString, line : UInt, flags : UInt32) -> Never).(closure #1).(closure #1) + 144
6    libswiftCore.dylib                 0x0000000100377760 specialized specialized String._withUnsafeBufferPointerToUTF8<A> ((UnsafeBufferPointer<UInt8>) throws -> A) throws -> A + 127
7    libswiftCore.dylib                 0x00000001004f1f40 partial apply for (_assertionFailure(StaticString, String, file : StaticString, line : UInt, flags : UInt32) -> Never).(closure #1) + 185
8    libswiftCore.dylib                 0x0000000100377250 specialized specialized StaticString.withUTF8Buffer<A> ((UnsafeBufferPointer<UInt8>) -> A) -> A + 342
9    libswiftCore.dylib                 0x00000001004a8130 specialized _assertionFailure(StaticString, String, file : StaticString, line : UInt, flags : UInt32) -> Never + 144
10   libswiftCore.dylib                 0x0000000100377140 assertionFailure(@autoclosure () -> String, file : StaticString, line : UInt) -> () + 96
11   Swindler                           0x000000010010f4f0 OSXStateDelegate.notifyInvalid() -> () + 88
12   Swindler                           0x000000010010f5f0 protocol witness for PropertyNotifier.notifyInvalid() -> () in conformance <A, B, C where ...> OSXStateDelegate<A, B, C> + 40
13   Swindler                           0x00000001000fd030 PropertyNotifierThunk.(init<A, B, C where ...> (A1, withEvent : B1.Type, receivingObject : C1.Type) -> PropertyNotifierThunk<A>).(closure #1) + 127
14   Swindler                           0x00000001000fd130 partial apply for PropertyNotifierThunk.(init<A, B, C where ...> (A1, withEvent : B1.Type, receivingObject : C1.Type) -> PropertyNotifierThunk<A>).(closure #1) + 136
15   Swindler                           0x00000001000f89a0 Property.handleError(Error) throws -> A.PropertyType + 1137
16   Swindler                           0x00000001000f6a00 Property.(init<A, B where ...> (A1, notifier : B1) -> Property<A>).(closure #2) + 191
17   Swindler                           0x00000001000f6bc0 partial apply for Property.(init<A, B where ...> (A1, notifier : B1) -> Property<A>).(closure #2) + 159
18   PromiseKit                         0x0000000100279750 State.(catch(on : DispatchQueue, policy : CatchPolicy, else : (Resolution<A>) -> (), execute : (Error) throws -> ()) -> ()).(closure #1).(closure #1) + 113
19   PromiseKit                         0x0000000100290a20 (contain_zalgo<A> (DispatchQueue, rejecter : (Resolution<A>) -> (), block : () throws -> ()) -> ()).(closure #1) + 137
20   PromiseKit                         0x000000010024fdc0 thunk + 39
21   libdispatch.dylib                  0x0000000100c57cf2 _dispatch_call_block_and_release + 12
22   libdispatch.dylib                  0x0000000100c4e784 _dispatch_client_callout + 8
23   libdispatch.dylib                  0x0000000100c5c95a _dispatch_main_queue_callback_4CF + 362
24   CoreFoundation                     0x00007fff8917abc0 __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__ + 9
25   CoreFoundation                     0x00007fff8913b370 __CFRunLoopRun + 2205
26   CoreFoundation                     0x00007fff8913af70 CFRunLoopRunSpecific + 420
27   HIToolbox                          0x00007fff8869bdcc RunCurrentEventLoopInMode + 240
28   HIToolbox                          0x00007fff8869bb41 ReceiveNextEventCommon + 432
29   HIToolbox                          0x00007fff8869badf _BlockUntilNextEventMatchingListInModeWithFilter + 71
30   AppKit                             0x00007fff86c345f4 _DPSNextEvent + 1120
31   AppKit                             0x00007fff873afd02 -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 2796
32   AppKit                             0x00007fff86c2903d -[NSApplication run] + 926
33   AppKit                             0x00007fff86bf3939 NSApplicationMain + 1237
34   TestProject                        0x000000010000f900 main + 13
35   libdyld.dylib                      0x00007fff9e8c3234 start + 1
(lldb)

Any thoughts how to overcome this?

Window ordering routines

Allow setting the window ordering, within overlapping windows or actual order on the screen, using AX APIs.

My main concern with this feature is performance and user experience when re-ordering is happening (What if they type something and it ends up in the wrong window? Should we capture all user input during the operation, and re-direct it to the window that is re-focused at the end of it? Maybe that's a further enhancement.) Also, handling timeouts and failure properly.

Hammerspoon supports sendToBack, only for "visible" ordering (currently overlapping windows). This might not be enough, though, because once a window is moved to overlap with this window it could show up behind the window that was just sent to the back. We should consider exposing both options for performance reasons if that use case is not needed.

Example app fails

Trying to run example app via running swift run and getting error:

/Users/.../tmp/Swindler/.build/checkouts/AXSwift/Sources/Observer.swift:38:19: error: thrown expression type 'AXError' does not conform to 'Error'
            throw error
                  ^~~~~

Some information about my system:

% swift -version
swift-driver version: 1.26.21 Apple Swift version 5.5.2 (swiftlang-1300.0.47.5 clang-1300.0.29.30)
Target: x86_64-apple-macosx11.0

Size property not immediately consistent with frame

size should delegate to frame on read, so we only have to update the size once when it changes, but can still atomically update the size.

In between reads, we can fire events where window.frame.size and window.size are observed not to match. (Probably not the end of the world, but not the level of consistency I want for Swindler.)

This is easily fixed with a custom property delegate.

"Safari Web Content (Cached)" processes causing 20 second+ initialization times

My Configuration:

  • Monterey 12.0.1
  • Safari 15.1

I have encountered very slow Swindler initialization times at times, and have traced it to "Safari Web Content (Cached)" processes. These NSRunningApplication have the activation policy NSApplicationActivationPolicyAccessory.

Any accessibility API query to retrieve an attribute from such a process throws an AXError.cannotComplete after 6 seconds. If there are many of such processes, and many attribute queries, Swindler can take minutes to initialize.

I have not been able to root cause this issue, as what's going on in AXUIElementCopyAttributeValue is a black box to me. This could be a bug in Apple's accessibility API, or who knows what.

One workaround is allowing the consumer to specify a filter for which applications to observer. Or a robust way of filtering out "Safari Web Content" processes, which AFAIK do not have any UI.

Swift Package Manager support?

I'm working on a project and looking forward to use Swindler. But I've already been using Swift Package Manager for quite a while.
Is that possible for Swindler to support installing with SPM in the future?

Mac App Store

Hi,
I'm just wondering if this library can be included in an App for the Mac App Store.
What Apple says in their guidelines about this? Can we use these APIs or we should stay inside of our sandbox? Thanks, regards.

Pascal

Swindler does not compile on Xcode 10.2

The new compiler seems more picky. I would try to fix it myself and my Swift Kung-Fu is not yet strong enough to see the fixes immediately. I tried setting the SWIFT_VERSION to 4.x, it does not help. Once you have upgraded to Xcode 10.2 you cannot compile Swindler any more.

I have attached the output of:

xcodebuild -workspace "Swindler.xcworkspace" -scheme "Swindler" | xcpretty

Mostly it complains about "parameter ... could not be inferred".

xcodebuild-xcpretty.log

Stop using class function `Application.all()`

Writing protocols and fakes that emulate AXSwift.Application is annoying, because .all() has to be reimplemented for every class in the hierarchy. We should pass in something that emulates NSWorkspace rather than depending on Application.all().

I'm not certain whether this will reduce boilerplate or not. Even if it's about the same level of complexity, it's probably a win because it makes internal patterns more consistent.

Animations

I'm liking the way Hammerspoon does animations. In short, it drives the animation from its own event loop, dispatching writes at a set interval (looks like it's 10ms or so). I wasn't sure if such an approach would work, but it looks pretty decent to me on my machine.

Some thoughts on implementation...

  • Animations should happen on a per-property basis.
  • While a property is being animated, we can keep the property's requestLock locked so no other reads/writes go through
    • This means no events on the property until the animation is done. external would be properly set.
  • Synchronous writes under a timer are probably the best approach. That way we don't pay the overhead of scheduling a function on the event loop.
    • On the other hand, if we are animating many properties at once, do we want many timers firing? Would only one be best? Many timers would provide "fairness" so that one application can't freeze out the rest, but it might be inefficient.
    • Perhaps one per application?? Pass an AnimationContext to each writeable property which contains the timer, and have one per application.
    • Also, don't forget that we can use timeout.
  • The core Animation class should simply provide the proper integration with the property internals. Allow a user to compute the actual values for each animation step. Include a simple "linear schedule" to get started. API should be similar to Cocoa animation APIs.
    • Is there a way to actually use the Cocoa animation schedules with our class?

Where is the event listening code?

My Objective C++ app has a minor need to listen to updates on window movement. For some reason, I can't find any documentation on the AX APIs and am interested in looking at the code for Swinder to see how I could implement it in my app. Alternatively, if it's possible to use swindler from Objective-C++ that would work too

How do I find the topmost window at a given point?

How would I find the topmost window at a point?
Here's the code I have, but I'm not sure how to figure out which window is on top.

I don't see any references to z-index in the existing or planned API docs.

import Cocoa
import Swindler
import PromiseKit

let currentCursor = NSEvent.mouseLocation
print(currentCursor)
let _ = Swindler.initialize()
    .done { state -> Void in
        print(
            state.knownWindows.filter({
                window in
                guard window.isMinimized.value else { return false }
                return window.frame.value.contains(currentCursor)
            }))
        exit(0)
    }.catch { error in
        print(error)
    }
CFRunLoopRun()

Merge position, size properties into rect

The reasoning behind this is that there are system user interactions that modify the position and size at the same time. Namely, resizing a window by dragging the top-left bottom-left corner (or bottom-right, or top-left). Surfacing this in the Swindler API should make for smoother handing of these interactions.

Don't receive ScreenLayoutChangedEvent events

Maybe I'm doing something wrong, but I don't receive ScreenLayoutChangedEvent events.

swindler.on { (event: ScreenLayoutChangedEvent) in
                print("screen changed") 
}

With the above if I change some parameter, for example the resolution, nothing is printed.

If I register manually (which is the same way Swindler does)

NotificationCenter.default.addObserver(forName: NSApplication.didChangeScreenParametersNotification,
                                               object: NSApplication.shared,
                                               queue: OperationQueue.main) {
                                                notification -> Void in
                                                print("screen parameters changed")}

it works.

Oh, and the example app in the project still uses the old API.

New windows aren't discovered on space change

We should really handle space changed events by scanning for unknown windows.

This should come with either a WindowDiscoveredEvent or a SpaceChangedEvent with a list of windows (or both).

Tracking issue for Spaces API

macOS doesn't expose much of the functionality of Spaces via public APIs, so Swindler's API surface will be pretty restricted. However, it should be good enough to support the basic functionality fo most window managers.

Those who want more can venture into private APIs on their own (they always break and I don't want to maintain that.)

Basically, we want to:

  • detect when the space changes
  • detect when we come back to a space that we've seen before
  • detect when we come to a new space that we haven't seen before
  • track which space each window is on

The first three goals (and part of the fourth) can be accomplished by subscribing to activeSpaceDidChangeNotification on NSWorkspace, then asking all our AXApplications for a list of windows. Only windows on the current space are returned. Finally, we can apply a heuristic, comparing lists of windows on each space to the list of windows on a current space.

Since we're using a heuristic, no space will have its own "identity" other than that of the windows which are on it. In particular, empty spaces are indistinguishable from each other, so we cull them when they are no longer active.

Mark writes that did not match the desired value as external events

When a property is written, we read back the new value to observe what changed, and fire an event with external = false to denote that the change was triggered by our program.

This can cause problems when something is happening simultaneously (i.e. the user is dragging/resizing the window). If the value we read back is not what we wrote, we should mark the event as external and let the program deal with it.

Sometimes, what we read is not what we wrote because what we wrote is out-of-bounds (i.e. moving the window off the screen). In that case, we should still mark the event as external so the program can see that something external (in this case, the application or operating system) affected the value.

Would we lose anything here? Is it ever important to know that an event occurred during a write? Maybe (I can't think of a reason off the top of my head), but we can always add that flag to events later.

Crash on setting frame - AXError.AttributeUnsupported

Hi,
I'm discovering your library in order to replace my current code which is currently using ScriptingBridge.

I'm trying to set the frame of the window, with code from SwindlerExample.
Here is the code i use :

    Swindler.initialize().then { state -> () in
        self.swindler = state
        self.setupEventHandlers()
        
        for (index, app) in self.swindler.runningApplications.enumerated() {
            print("App: \(index): \(app)")

            if (app.bundleIdentifier == "com.apple.finder") {
                let finderApp = app;
                print("Desc: \(finderApp.description)")
                
                let finderWindow = finderApp.knownWindows.first
                
                finderWindow?.size.set(CGSize(width: 100, height: 100)) //working
                finderWindow?.frame.set(CGRect(origin: CGPoint(x: 100, y: 100), size: CGSize(width: 300, height: 300))) //crashes with invalid attribute error (when debugging, attribute is set to role…)
            }
        }

    }.catch { error in
        print("Fatal error: failed to initialize Swindler: \(error)")
        NSApp.terminate(self)
    }

Am I doing something wrong ? Setting size works correctly. As I want to set the position, I'm using frame (according to the doc), but AXSwift.Attribute is always set to role… Maybe I've made a stupid mistake.

Thanks for your help :)

Move window to another desktop

I could not find in the documentation
how to move a window from one desktop to another?

I can do window.frame.set(...
But this only works within the current desktop

Consider moving handleScreenChange off main thread

Based on discussion in #44. Trace the performance of this function and/or just move it to another thread.

Since we still have to invoke the Swindler event on the main thread, I was worried about "unnecessary overhead" before, but I think that's negligible for a rare event compared to a potential main thread freeze.

Segmentation fault when building in Xcode 7.3.1

AXSwift build fines but Swindler fails:
Command failed due to signal: Segmentation fault: 11

CompileSwift normal x86_64 /Users/michael/src/Swindler/Swindler/Successes+PromiseKit.swift
    cd /Users/michael/src/Swindler
    /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swift -frontend -c /Users/michael/src/Swindler/Swindler/AXPropertyDelegate.swift /Users/michael/src/Swindler/Swindler/Window.swift /Users/michael/src/Swindler/Swindler/Log.swift /Users/michael/src/Swindler/Swindler/AXSwiftProtocols.swift /Users/michael/src/Swindler/Swindler/Property.swift /Users/michael/src/Swindler/Swindler/Screen.swift -primary-file /Users/michael/src/Swindler/Swindler/Successes+PromiseKit.swift /Users/michael/src/Swindler/Swindler/Errors.swift /Users/michael/src/Swindler/Swindler/State.swift /Users/michael/src/Swindler/Swindler/Events.swift /Users/michael/src/Swindler/Swindler/Application.swift -target x86_64-apple-macosx10.10 -enable-objc-interop -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk -I /Users/michael/Library/Developer/Xcode/DerivedData/Swindler-gzxauzymqixryzcamosildqhpuio/Build/Products/Debug -I /Users/michael/src/Swindler/Swindler -F /Users/michael/Library/Developer/Xcode/DerivedData/Swindler-gzxauzymqixryzcamosildqhpuio/Build/Products/Debug -enable-testing -g -import-underlying-module -module-cache-path /Users/michael/Library/Developer/Xcode/DerivedData/ModuleCache -D COCOAPODS -D COCOAPODS -D SWINDLER_DEBUG -serialize-debugging-options -Xcc -I/Users/michael/Library/Developer/Xcode/DerivedData/Swindler-gzxauzymqixryzcamosildqhpuio/Build/Intermediates/Swindler.build/Debug/Swindler.build/swift-overrides.hmap -Xcc -iquote -Xcc /Users/michael/Library/Developer/Xcode/DerivedData/Swindler-gzxauzymqixryzcamosildqhpuio/Build/Intermediates/Swindler.build/Debug/Swindler.build/Swindler-generated-files.hmap -Xcc -I/Users/michael/Library/Developer/Xcode/DerivedData/Swindler-gzxauzymqixryzcamosildqhpuio/Build/Intermediates/Swindler.build/Debug/Swindler.build/Swindler-own-target-headers.hmap -Xcc -I/Users/michael/Library/Developer/Xcode/DerivedData/Swindler-gzxauzymqixryzcamosildqhpuio/Build/Intermediates/Swindler.build/Debug/Swindler.build/Swindler-all-non-framework-target-headers.hmap -Xcc -ivfsoverlay -Xcc /Users/michael/Library/Developer/Xcode/DerivedData/Swindler-gzxauzymqixryzcamosildqhpuio/Build/Intermediates/Swindler.build/all-product-headers.yaml -Xcc -iquote -Xcc /Users/michael/Library/Developer/Xcode/DerivedData/Swindler-gzxauzymqixryzcamosildqhpuio/Build/Intermediates/Swindler.build/Debug/Swindler.build/Swindler-project-headers.hmap -Xcc -I/Users/michael/Library/Developer/Xcode/DerivedData/Swindler-gzxauzymqixryzcamosildqhpuio/Build/Products/Debug/include -Xcc -I/Users/michael/Library/Developer/Xcode/DerivedData/Swindler-gzxauzymqixryzcamosildqhpuio/Build/Intermediates/Swindler.build/Debug/Swindler.build/DerivedSources/x86_64 -Xcc -I/Users/michael/Library/Developer/Xcode/DerivedData/Swindler-gzxauzymqixryzcamosildqhpuio/Build/Intermediates/Swindler.build/Debug/Swindler.build/DerivedSources -Xcc -DDEBUG=1 -Xcc -DCOCOAPODS=1 -Xcc -working-directory/Users/michael/src/Swindler -emit-module-doc-path /Users/michael/Library/Developer/Xcode/DerivedData/Swindler-gzxauzymqixryzcamosildqhpuio/Build/Intermediates/Swindler.build/Debug/Swindler.build/Objects-normal/x86_64/Successes+PromiseKit~partial.swiftdoc -Onone -module-name Swindler -emit-module-path /Users/michael/Library/Developer/Xcode/DerivedData/Swindler-gzxauzymqixryzcamosildqhpuio/Build/Intermediates/Swindler.build/Debug/Swindler.build/Objects-normal/x86_64/Successes+PromiseKit~partial.swiftmodule -serialize-diagnostics-path /Users/michael/Library/Developer/Xcode/DerivedData/Swindler-gzxauzymqixryzcamosildqhpuio/Build/Intermediates/Swindler.build/Debug/Swindler.build/Objects-normal/x86_64/Successes+PromiseKit.dia -emit-dependencies-path /Users/michael/Library/Developer/Xcode/DerivedData/Swindler-gzxauzymqixryzcamosildqhpuio/Build/Intermediates/Swindler.build/Debug/Swindler.build/Objects-normal/x86_64/Successes+PromiseKit.d -emit-reference-dependencies-path /Users/michael/Library/Developer/Xcode/DerivedData/Swindler-gzxauzymqixryzcamosildqhpuio/Build/Intermediates/Swindler.build/Debug/Swindler.build/Objects-normal/x86_64/Successes+PromiseKit.swiftdeps -o /Users/michael/Library/Developer/Xcode/DerivedData/Swindler-gzxauzymqixryzcamosildqhpuio/Build/Intermediates/Swindler.build/Debug/Swindler.build/Objects-normal/x86_64/Successes+PromiseKit.o

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/module.modulemap:1:8: error: redefinition of module 'Compression'
module Compression [system] [extern_c] {
       ^
/usr/include/module.modulemap:1:8: note: previously defined here
module Compression [system] [extern_c] {
       ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/module.modulemap:6:8: error: redefinition of module 'Darwin'
module Darwin [system] [extern_c] {
       ^
/usr/include/module.modulemap:6:8: note: previously defined here
module Darwin [system] [extern_c] {
       ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/module.modulemap:1599:8: error: redefinition of module 'os'
module os [system] [extern_c] {
       ^
/usr/include/module.modulemap:1599:8: note: previously defined here
module os [system] [extern_c] {
       ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/module.modulemap:1615:8: error: redefinition of module 'libkern'
module libkern [system] [extern_c] {
       ^
/usr/include/module.modulemap:1615:8: note: previously defined here
module libkern [system] [extern_c] {
       ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/module.modulemap:1640:8: error: redefinition of module 'ldap'
module ldap [system] [extern_c] {
       ^
/usr/include/module.modulemap:1640:8: note: previously defined here
module ldap [system] [extern_c] {
       ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/module.modulemap:1673:8: error: redefinition of module 'krb5'
module krb5 [system] [extern_c] {
       ^
/usr/include/module.modulemap:1673:8: note: previously defined here
module krb5 [system] [extern_c] {
       ^
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/module.modulemap:1700:8: error: redefinition of module 'XPC'
module XPC [system] [extern_c] {
       ^
/usr/include/module.modulemap:1700:8: note: previously defined here
module XPC [system] [extern_c] {
       ^
0  swift                    0x000000010957c66b llvm::sys::PrintStackTrace(llvm::raw_ostream&) + 43
1  swift                    0x000000010957b956 llvm::sys::RunSignalHandlers() + 70
2  swift                    0x000000010957cccf SignalHandler(int) + 287
3  libsystem_platform.dylib 0x00007fff9ba3752a _sigtramp + 26
4  libsystem_platform.dylib 0x00007000008002c0 _sigtramp + 1692175792
5  swift                    0x0000000107d4c5f6 clang::Preprocessor::MacroState::getModuleInfo(clang::Preprocessor&, clang::IdentifierInfo const*) const + 198
6  swift                    0x0000000107d4bfe6 clang::Preprocessor::getMacroDefinition(clang::IdentifierInfo const*) + 326
7  swift                    0x00000001086e18ca clang::Preprocessor::HandleIdentifier(clang::Token&) + 298
8  swift                    0x0000000108686b26 clang::Lexer::LexIdentifier(clang::Token&, char const*) + 262
9  swift                    0x000000010868dc08 clang::Lexer::LexTokenInternal(clang::Token&, bool) + 7864
10 swift                    0x00000001086e1f14 clang::Preprocessor::Lex(clang::Token&) + 68
11 swift                    0x00000001086b1ec7 clang::Preprocessor::ReadMacroName(clang::Token&, clang::MacroUse, bool*) + 55
12 swift                    0x00000001086b8516 clang::Preprocessor::HandleDefineDirective(clang::Token&, bool) + 54
13 swift                    0x00000001086b4362 clang::Preprocessor::HandleDirective(clang::Token&) + 1778
14 swift                    0x000000010868e6f1 clang::Lexer::LexTokenInternal(clang::Token&, bool) + 10657
15 swift                    0x00000001086e1f14 clang::Preprocessor::Lex(clang::Token&) + 68
16 swift                    0x0000000107fb5088 clang::Parser::ParseTopLevelDecl(clang::OpaquePtr<clang::DeclGroupRef>&) + 232
17 swift                    0x0000000107f35c75 clang::ParseAST(clang::Sema&, bool, bool) + 501
18 swift                    0x0000000107d75550 clang::FrontendAction::Execute() + 64
19 swift                    0x0000000107d42242 clang::CompilerInstance::ExecuteAction(clang::FrontendAction&) + 930
20 swift                    0x000000010951c5a0 llvm::CrashRecoveryContext::RunSafely(llvm::function_ref<void ()>) + 272
21 swift                    0x000000010951c6f0 RunSafelyOnThread_Dispatch(void*) + 48
22 swift                    0x000000010957d97d ExecuteOnThread_Dispatch(void*) + 13
23 libsystem_pthread.dylib  0x00007fff927b799d _pthread_body + 131
24 libsystem_pthread.dylib  0x00007fff927b791a _pthread_body + 0
25 libsystem_pthread.dylib  0x00007fff927b5351 thread_start + 13
Stack dump:
0.  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/sys/_symbol_aliasing.h:35:2: current parser token 'define'```

Flaky spec: AXUIElement_notifications__when_a_property_value_changes_right_after_reading_it__is_updated_correctly

https://travis-ci.org/tmandry/Swindler/builds/104360471

All other tests ran in <25ms, so I doubt this taking >1000ms was a fluke.

Relevant log:

Failures:
  0) -[OSXWindowDelegateNotificationSpec AXUIElement_notifications__when_a_property_value_changes_right_after_reading_it__is_updated_correctly_UserstravisbuildtmandrySwindlerSwindlerTestsWindowSpecswift_194] (SwindlerTests.xctest)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Jan 23 23:35:47  xctest[1172] <Debug>: Notification WindowMiniaturized on unknown element SwindlerTests.AdversaryWindowElement, deferring
/Users/travis/build/tmandry/Swindler/SwindlerTests/WindowSpec.swift:207: failed - expected to eventually be true, got <false>
:
204           }
205 
206           return initialize().then { winDelegate -> () in
207             expect(winDelegate.isMinimized.value).toEventually(beTrue())
                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
208           }
209         }
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Note the debug log which might be important.

https://travis-ci.org/tmandry/Swindler/builds/103275152

Failures:
  0) -[OSXWindowDelegateNotificationSpec AXUIElement_notifications__when_a_property_value_changes_right_after_reading_it__is_updated_correctly_UserstravisbuildtmandrySwindlerSwindlerTestsWindowSpecswift_194] (SwindlerTests.xctest)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Jan 19 07:01:22  xctest[1229] <Debug>: addNotification(WindowCreated) on SwindlerTests.TestApplicationElement responded with () in 0.0ms
Jan 19 07:01:22  xctest[1229] <Debug>: addNotification(MainWindowChanged) on SwindlerTests.TestApplicationElement responded with () in 0.0ms
Jan 19 07:01:22  xctest[1229] <Debug>: addNotification(FocusedWindowChanged) on SwindlerTests.TestApplicationElement responded with () in 0.0ms
Jan 19 07:01:22  xctest[1229] <Debug>: addNotification(ApplicationHidden) on SwindlerTests.TestApplicationElement responded with () in 0.0ms
Jan 19 07:01:22  xctest[1229] <Debug>: addNotification(ApplicationShown) on SwindlerTests.TestApplicationElement responded with () in 0.0ms
Jan 19 07:01:22  xctest[1229] <Debug>: arrayAttribute(Windows) on SwindlerTests.TestApplicationElement responded with Optional([SwindlerTests.AdversaryWindowElement]) in 0.0ms
Jan 19 07:01:22  xctest[1229] <Debug>: getMultipleAttributes([AXSwift.Attribute.MainWindow, AXSwift.Attribute.FocusedWindow, AXSwift.Attribute.Hidden]) on SwindlerTests.TestApplicationElement responded with [AXSwift.Attribute.Hidden: false] in 0.0ms
Jan 19 07:01:22  xctest[1229] <Debug>: addNotification(TitleChanged) on SwindlerTests.AdversaryWindowElement responded with () in 0.0ms
Jan 19 07:01:22  xctest[1229] <Debug>: addNotification(Resized) on SwindlerTests.AdversaryWindowElement responded with () in 0.0ms
Jan 19 07:01:22  xctest[1229] <Debug>: addNotification(WindowMiniaturized) on SwindlerTests.AdversaryWindowElement responded with () in 0.1ms
Jan 19 07:01:22  xctest[1229] <Debug>: addNotification(Moved) on SwindlerTests.AdversaryWindowElement responded with () in 0.0ms
Jan 19 07:01:22  xctest[1229] <Debug>: addNotification(WindowDeminiaturized) on SwindlerTests.AdversaryWindowElement responded with () in 0.0ms
Jan 19 07:01:22  xctest[1229] <Debug>: addNotification(UIElementDestroyed) on SwindlerTests.AdversaryWindowElement responded with () in 0.0ms
Jan 19 07:01:22  xctest[1229] <Debug>: Received WindowMiniaturized on SwindlerTests.AdversaryWindowElement
Jan 19 07:01:22  xctest[1229] <Debug>: Notification WindowMiniaturized on unknown element SwindlerTests.AdversaryWindowElement, deferring
Jan 19 07:01:22  xctest[1229] <Debug>: getMultipleAttributes([AXSwift.Attribute.Position, AXSwift.Attribute.Size, AXSwift.Attribute.Title, AXSwift.Attribute.Minimized, AXSwift.Attribute.FullScreen, AXSwift.Attribute.Subrole]) on SwindlerTests.AdversaryWindowElement responded with [AXSwift.Attribute.Size: (0.0, 0.0), AXSwift.Attribute.Position: (0.0, 0.0), AXSwift.Attribute.Title: "Window 140", AXSwift.Attribute.Minimized: false, AXSwift.Attribute.FullScreen: false] in 1.1ms
/Users/travis/build/tmandry/Swindler/SwindlerTests/WindowSpec.swift:207: failed - expected to eventually be true, got <false>
:
204           }
205 
206           return initialize().then { winDelegate -> () in
207             expect(winDelegate.isMinimized.value).toEventually(beTrue())
                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
208           }
209         }
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Configurable dispatch queue for event handlers

So we aren't clogging up the main thread with our event handlers.

Do we want to support queues with multiple threads? If so I'll have to spend some time thinking about proper synchronization here, because I'm not 100% sure if the window and application delegate code is thread-safe. Properties definitely are, though.

See also discussion in #44.

Notification windows are not trackable and cause swindler to print errors

When running an app that uses swindler I get the following whenever the NotoficationCenter displays a message:

Cannot convert property value nil to type String
Cannot convert property value nil to type Bool
Cannot convert property value nil to type Bool

Plus the windows created by NotoficationCenter are not visible/trackable in swindler.

Steps to reproduce - Start a swinder-enabled app and run

osascript -e 'display notification "Lorem ipsum dolor sit amet" with title "Title"'

to produce a notification from Terminal.

Expected behaviour would be not to get those error messages and being able to track the windows created by NotificationCenter.

Window.raise()

Add a raise() function to Window.

It should be easy to check if the window is on the current space (TODO), so raise() should move to another space if the window is there.

Note that Hammerspoon has a workaround for dealing with Finder windows when moving between spaces. This probably needs to be recreated (with promises, not timers!)

Add Close Button to Window element

For my use case at least,
The ability to fullscreen, minimize easily is great. But I miss the ability also to close a window.
I guess it can be easily added. If you'd prefer a PR, let me know.

Document how to use with Cocoapods

0.1.0 will be the first version actually published on Cocoapods. Need to add something to the README about adding it to your Podfile.

Consider making non-Promise property access prettier

It’s likely that all property reads and many writes will be done without promises. We can “split” the API into promise and non-promise based further up in the chain, to something like the following:

window.position = CGRect(x: 100, y: 200)
print(window.title)

window.async.size.set(CGSize(width: 300, height: 400)).then { ... }

The only problem with the above is that async implies that non-Promise accesses are performed synchronously (really, they are queued onto a background thread like everything else).

Resize doesn't capture new position

With this setup, I don't see any updated value output while dragging from the edge of a window to resize it. The size changes, but the position isn't updated.

image

Cannot be added as a Swift Package

I tried to add Swindler as a package via Xcode as per their docs.

Due to AXSwift being a locally referenced package, it's not possible to do so.

image

I forked and created a branch that sets the path correctly and it installs correctly into Xcode - https://github.com/chrispaynter/Swindler/tree/hotfix/change-axswift-reference

I'm guessing you're using submodules for ease of debugging / development of AXSwift.

Might be good to add some extra documentation to the readme about setting up Swindler as a dependency.

Understandably, Swindler is still alpha. Might be worth looking at what versioned (non master branch) package references might look like moving forward.

Support manipulating window tabs

The AX APIs make it pretty easy to manipulate tabs in windows. I can see this being useful in x3 e.g. for setting a "mark" on a browser tab, rather than just the browser window itself.

This might belong in another library as sort of an extension to Swindler. I'm not sure if there's any app-specific stuff that goes into it, but it does feel sort of outside the core functionality of Swindler that I would want to maintain.

For reference see Hammerspoon implementation.

API docs

Generate API docs (probably with jazzy) and host them somewhere (probably GitHub pages).

Doesn't keep track of newly opened applications

Swindler doesn't keep track of new opened applications. The "runningApplications" in State isn't updating and things like "frontmostApplication" can be nil if the application is launched after Swindlers initialization.

To reproduce:
Open the included SwindlerExample. Open another app. "FrontmostApplicationChangedEvent" gets called but frontmostApplication is nil.

Use consistent coordinate system for all of Swindler

Currently, Screen uses a coordinate system with origin at the bottom-left, while Window uses an origin at the top-left. This is a weird quirk of macOS.

We should probably switch to using the same system as Screen everywhere, since that is what is used in the rest of Cocoa, but it would be a really invasive change.

Mouse over active window title bar event

I’d like to have an event that is raised when my mouse hovers over the title bar of the active window. I’m trying to build a window management app that can use vim like commands “jkl;” to snap the window to the right places. I don’t see an existing event that can help with that. Any advice?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.