Code Monkey home page Code Monkey logo

screenshotty's Introduction

Screenshotty

The library combines MediaProjection, PixelCopy and Canvas drawing and provides an easy-to-use API, abstracted from the Android framework and the complexities of the underlying mechanisms, to capture precisely what a user sees on their screen.

The sample app shows how to use the library.

Gradle

Add this to your dependencies block.

implementation 'eu.bolt:screenshotty:1.0.4'

To use a reactive wrapper also add:

implementation 'eu.bolt:screenshotty-rx:1.0.4'

To work with coroutines add:

implementation 'eu.bolt:screenshotty-coroutines:1.0.4'

Wiki

General

If we want to capture a screenshot inside the app, the simplest approach is to draw the root view on a Bitmap, but this approach won't work correctly if there are open dialogs, or view hierarchy contains maps or other SurfaceViews. Screenshotty uses PixelCopy and MediaProjection to provide the correct image in all these cases.

First the library tries to make a PixelCopy with dialogs, retrieved via reflection, rendered on top.

If this approach fails, user will see a record screen permission dialog. Screenshotty minimizes the number of times the dialog is shown: permission has to be granted only once per process lifetime. If "Don't show again" option (removed in Android 10) is checked, the system will remember user's choice for all the future invocations. If the permission is granted, a MediaProjection API is used to take a single frame and provide it to result listeners.

In case MediaProjection fails, fallback strategies are invoked one-by-one until the first one succeeds to provide a Bitmap.

Usage

  1. Create a ScreenshotManager:
screenshotManager = ScreenshotManagerBuilder(this)
   .withCustomActionOrder(ScreenshotActionOrder.pixelCopyFirst()) //optional, ScreenshotActionOrder.pixelCopyFirst() by default
   .withPermissionRequestCode(REQUEST_SCREENSHOT_PERMISSION) //optional, 888 by default
   .build()
  1. Make sure the object receives activity results:
override fun onActivityResult(requestCode: Int, resultCode: Int, data: Intent?) {
   super.onActivityResult(requestCode, resultCode, data)
   screenshotManager.onActivityResult(requestCode, resultCode, data)
}
  1. Request a screenshot and observe the result:
val screenshotResult = screenshotManager.makeScreenshot()
val subscription = screenshotResult.observe(
   onSuccess = { processScreenshot(it) },
   onError = { onMakeScreenshotFailed(it) }
)
  1. If you're no longer interested in the result (as when your Activity is destroyed), you can unsubscribe your observers using the object you got from observe().
override fun onDestroy() {
   super.onDestroy()
   subscription.dispose()
}

Working with result

When you receive a Screenshot you can either get a Bitmap object from it:

fun show(screenshot: Screenshot) {
   val bitmap = when (screenshot) {
      is ScreenshotBitmap -> screenshot.bitmap
   }
   screenshotPreview.setImageBitmap(bitmap)
}

Or use ScreenshotFileSaver provided by the library to write the image to a file:

fun writeToFile(screenshot: Screenshot): File {
   val fileSaver = ScreenshotFileSaver.create(Bitmap.CompressFormat.PNG)
   val targetFile = File(context.filesDir, "screenshot")
   fileSaver.saveToFile(targetFile, screenshot)
   return targetFile
}

Reactive wrapper

If you're using screenshotty-rx, you can transform your ScreenshotManager object into RxScreenshotManager:

val rxScreenshotManager = screenshotManager.asRxScreenshotManager() //or RxScreenshotWrapper.wrap(screenshotManager)

Usage is exactly the same, but makeScreenshot() returns Single<Screenshot> instead of ScreenshotResult, so you can use all the expressive power of reactive composition to process the result:

subscription = rxScreenshotManager.makeScreenshot()
   .observeOn(Schedulers.io())
   .map(::writeToFile)
   .doOnSuccess(::sendScreenshotFile)
   .observeOn(AndroidSchedulers.mainThread())
   .subscribe(
      onSuccess = ::onScreenshotSent,
      onError = ::handleError
   )

Coroutines wrapper

Screenshooty also supports coroutines by exposing the ScreenshotManager.makeScreenshotAsync() suspend extension function.

    screenshotManager.makeScreenshotAsync()

Usage is exactly the same, but makeScreenshot() is a suspend function.

Fallback strategies

When constructing a ScreenshotManager you can add any number of objects that implement FallbackStrategy interface. If PixelCopy or MediaProjection fails for some reason, fallback strategies will be invoked one by one in the order they were added, until the first one succeeds to provide a Bitmap.

If no strategies were added or all of them failed, the default one (that simply calls draw on the root view and tries to render dialogs retrieved via reflection on top) will be invoked.

Actions order

When constructing a ScreenshotManager an order in which screenshot actions will be taken can be specified. By default the order is PixelCopy -> MediaProjection -> Fallbacks, but that can be easily changed.
Note that Fallbacks will still run in the order they were added in. ScreenshotActionOrder provides all possible actions and convenient order creators.

License

MIT License

Copyright (c) 2020 Bolt Technologies OÜ

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

screenshotty's People

Contributors

aleksei-sivkov avatar jan-stoltman-bolt avatar xzaleksey avatar yarolegovich avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

screenshotty's Issues

Cannot get screenshot in background activity

screenshotManager = new ScreenshotManagerBuilder(getActivity())
        .withPermissionRequestCode(REQUEST_SCREENSHOT_PERMISSION)
        .build();
ScreenshotResult screenshotResult = screenshotManager.makeScreenshot();
// following code seems no usage
screenshotResult.observe((screenshot) -> {
    // WON'T GET THE SCREENSHOT 
    return Unit.INSTANCE;
}, (throwable -> {
    return Unit.INSTANCE;
}));

If the activity is in foreground, it performs well. However, when broadcasting the activity, and put this code in receiver's callback, I cannot get the screenshot.

Cannot get screenshot if activity is in background.

screenshotManager = new ScreenshotManagerBuilder(getActivity())
        .withPermissionRequestCode(REQUEST_SCREENSHOT_PERMISSION)
        .build();
ScreenshotResult screenshotResult = screenshotManager.makeScreenshot();
// following code seems no usage
screenshotResult.observe((screenshot) -> {
    // WON'T GET THE SCREENSHOT 
    return Unit.INSTANCE;
}, (throwable -> {
    return Unit.INSTANCE;
}));

If the activity is in foreground, it performs well. However, when broadcasting the activity, and put this code in receiver's callback, I cannot get the screenshot.

Handler (android.os.Handler) {2d4151b9} sending message to a Handler on a dead thread

After makeScreenshot() I got this warnings in logcat

11-03 20:02:21.094 13372-13389/com.test.app W/MessageQueue: Handler (android.os.Handler) {77bc449} sending message to a Handler on a dead thread
    java.lang.IllegalStateException: Handler (android.os.Handler) {77bc449} sending message to a Handler on a dead thread
        at android.os.MessageQueue.enqueueMessage(MessageQueue.java:325)
        at android.os.Handler.enqueueMessage(Handler.java:631)
        at android.os.Handler.sendMessageAtTime(Handler.java:600)
        at android.os.Handler.sendMessageDelayed(Handler.java:570)
        at android.os.Handler.post(Handler.java:326)
        at android.media.projection.MediaProjection$CallbackRecord.onStop(MediaProjection.java:204)
        at android.media.projection.MediaProjection$MediaProjectionCallback.onStop(MediaProjection.java:189)
        at android.media.projection.IMediaProjectionCallback$Stub.onTransact(IMediaProjectionCallback.java:49)
        at android.os.Binder.execTransact(Binder.java:446)
11-03 20:02:21.096 13372-13887/com.test.app W/MessageQueue: Handler (android.os.Handler) {b4db7bf} sending message to a Handler on a dead thread
    java.lang.IllegalStateException: Handler (android.os.Handler) {b4db7bf} sending message to a Handler on a dead thread
        at android.os.MessageQueue.enqueueMessage(MessageQueue.java:325)
        at android.os.Handler.enqueueMessage(Handler.java:631)
        at android.os.Handler.sendMessageAtTime(Handler.java:600)
        at android.os.Handler.sendMessageDelayed(Handler.java:570)
        at android.os.Handler.post(Handler.java:326)
        at android.media.projection.MediaProjection$CallbackRecord.onStop(MediaProjection.java:204)
        at android.media.projection.MediaProjection$MediaProjectionCallback.onStop(MediaProjection.java:189)
        at android.media.projection.IMediaProjectionCallback$Stub.onTransact(IMediaProjectionCallback.java:49)
        at android.os.Binder.execTransact(Binder.java:446)
11-03 20:02:21.138 13372-13886/com.test.app W/MessageQueue: Handler (android.os.Handler) {38257813} sending message to a Handler on a dead thread
    java.lang.IllegalStateException: Handler (android.os.Handler) {38257813} sending message to a Handler on a dead thread
        at android.os.MessageQueue.enqueueMessage(MessageQueue.java:325)
        at android.os.Handler.enqueueMessage(Handler.java:631)
        at android.os.Handler.sendMessageAtTime(Handler.java:600)
        at android.os.Handler.sendMessageDelayed(Handler.java:570)
        at android.os.Handler.post(Handler.java:326)
        at android.media.projection.MediaProjection$CallbackRecord.onStop(MediaProjection.java:204)
        at android.media.projection.MediaProjection$MediaProjectionCallback.onStop(MediaProjection.java:189)
        at android.media.projection.IMediaProjectionCallback$Stub.onTransact(IMediaProjectionCallback.java:49)
        at android.os.Binder.execTransact(Binder.java:446)
11-03 20:02:21.186 13372-13389/com.test.app W/MessageQueue: Handler (android.os.Handler) {ee0bb7f} sending message to a Handler on a dead thread
    java.lang.IllegalStateException: Handler (android.os.Handler) {ee0bb7f} sending message to a Handler on a dead thread
        at android.os.MessageQueue.enqueueMessage(MessageQueue.java:325)
        at android.os.Handler.enqueueMessage(Handler.java:631)
        at android.os.Handler.sendMessageAtTime(Handler.java:600)
        at android.os.Handler.sendMessageDelayed(Handler.java:570)
        at android.os.Handler.post(Handler.java:326)
        at android.media.projection.MediaProjection$CallbackRecord.onStop(MediaProjection.java:204)
        at android.media.projection.MediaProjection$MediaProjectionCallback.onStop(MediaProjection.java:189)
        at android.media.projection.IMediaProjectionCallback$Stub.onTransact(IMediaProjectionCallback.java:49)
        at android.os.Binder.execTransact(Binder.java:446)

PixelCopy delegate doesn't render dialogs

Dialogs and popup windows have their own Window, so the screenshot we get by making a PixelCopy of activity window won't capture their views.
Since the probability that a dialog/popup contains a SurfaceView is very low, the proposal is to simply render dialogs as Views on top of the Bitmap we get from PixelCopy.
For this, a separate class that renders floating panels can be created and we can also reuse it in DefaultFallbackStrategy to remove Falcon from the project. What I don't like is that Falcon contains some unobvious logic of positioning dialogs on top of activity in the list and also draws all the activity stack stored in global window manager.
The only thing I'm not sure about is whether to make dialogs rendering optional or mandatory. I personally favor the former, because if for some reason reflection fails for an activity without dialogs we just discard a perfectly valid screenshot. What do you think about this?

PixelCopy BMP sizes are wrong

The ScreenShotSpec taken from the window now uses the getSize method, that removes the soft buttons from the size, however the screenshot contains the part where it should be (in black) and the status bar as well. This makes the aspect ratio wrong. You should use getRealSize instead.

version 1.0.3 on Android 10 using PixelCopy API fails, reverting to 1.0.0 uses MediaProjection and works

on Android 10 (Samsung A9) using the latest version with the following code you get a black screenshot:

public class ScreenshotActivity extends Activity {
    private ScreenshotManager _mgr = null;
    private ScreenshotResult _scrRes = null;

    @Override
    protected void onActivityResult(int requestCode, int resultCode, @Nullable Intent data) {
        // ----> THIS IS NEVER CALLED
        super.onActivityResult(requestCode, resultCode, data);
        _mgr.onActivityResult(requestCode, resultCode, data);
        finishActivity(requestCode);
    }

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);

        // setup screenshotty
        ScreenshotManagerBuilder bld = new ScreenshotManagerBuilder(this);
        _mgr = bld.build();
        _scrRes = _mgr.makeScreenshot();

        // observe the result
        _scrRes.observe((result) -> {
            // ----> THIS IS CALLED, BUT THE RESULTING SCREENSHOT IS ALL BLACK
            Screenshot scr = (Screenshot)result;
            saveScreenshot(scr);
            finish();
            return Unit.INSTANCE;
        }, (ex -> {
            // failure
            finish();
            return Unit.INSTANCE;
        }));
    }
}

after a few digging around, i think it's the new Pixel API which fails somehow (but still report success), and that is used as default on android 10. as you can see from the comment, the observe successful callback is called without the OnActivityResult() handler called first, thus resulting in the black screenshot.

reverting to version 1.0.0 (which uses the mediaprojection api) works correctly, btw .... still, the PixelCopy API implementation is probably flawed/wrong/not applicable.
i would suggest to offer a way to select the API to use instead, without defaulting 'blindly'.... unless until you find a workaround.

cheers!

Taking 2sec to create a screenshot, can this time be reduced ?

Hey, This Library is great and working completely fine but with one little problem, that sometimes it gives error " Error: BufferQueue has been abandoned." inbetween of app process. However this logcat error does not stop my app, but slows it down. And it takes approx 2 sec to makescreenshot and give the result but I need it to run within milliseconds so that other process can be done. Thanks In Advance.

Setting `ScreenshotActionOrder.mediaProjectionFirst()` crashes the app.

I followed the set up according to your README. Works great if ScreenshotActionOrder.pixelCopyFirst() is used. However, I need to support devices older than SDK 24. So, I tested setting ScreenshotActionOrder.mediaProjectionFirst() and it caused my app to crash.

Below is the crash log:

2021-10-12 17:37:32.318 12146-12146/io.github.rexmtorres.android.screenshot E/AndroidRuntime: FATAL EXCEPTION: main
    Process: io.github.rexmtorres.android.screenshot, PID: 12146
    java.lang.RuntimeException: Failure delivering result ResultInfo{who=null, request=1, result=-1, data=Intent { (has extras) }} to activity {io.github.rexmtorres.android.screenshot/io.github.rexmtorres.android.screenshot.MainActivity}: java.lang.SecurityException: Media projections require a foreground service of type ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PROJECTION
        at android.app.ActivityThread.deliverResults(ActivityThread.java:5304)
        at android.app.ActivityThread.handleSendResult(ActivityThread.java:5343)
        at android.app.servertransaction.ActivityResultItem.execute(ActivityResultItem.java:54)
        at android.app.servertransaction.ActivityTransactionItem.execute(ActivityTransactionItem.java:45)
        at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135)
        at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2214)
        at android.os.Handler.dispatchMessage(Handler.java:106)
        at android.os.Looper.loopOnce(Looper.java:201)
        at android.os.Looper.loop(Looper.java:288)
        at android.app.ActivityThread.main(ActivityThread.java:7842)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:548)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1003)
     Caused by: java.lang.SecurityException: Media projections require a foreground service of type ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PROJECTION
        at android.os.Parcel.createExceptionOrNull(Parcel.java:2425)
        at android.os.Parcel.createException(Parcel.java:2409)
        at android.os.Parcel.readException(Parcel.java:2392)
        at android.os.Parcel.readException(Parcel.java:2334)
        at android.media.projection.IMediaProjection$Stub$Proxy.start(IMediaProjection.java:235)
        at android.media.projection.MediaProjection.<init>(MediaProjection.java:59)
        at android.media.projection.MediaProjectionManager.getMediaProjection(MediaProjectionManager.java:119)
        at eu.bolt.screenshotty.internal.projection.MediaProjectionDelegateV21.getMediaProjection(MediaProjectionDelegateV21.kt:162)
        at eu.bolt.screenshotty.internal.projection.MediaProjectionDelegateV21.onActivityResult(MediaProjectionDelegateV21.kt:74)
        at eu.bolt.screenshotty.internal.ScreenshotManagerImpl.onActivityResult(ScreenshotManagerImpl.kt:41)
        at io.github.rexmtorres.android.screenshot.MainActivity.onActivityResult(MainActivity.kt:41)
        at android.app.Activity.dispatchActivityResult(Activity.java:8382)
        at android.app.ActivityThread.deliverResults(ActivityThread.java:5297)
        at android.app.ActivityThread.handleSendResult(ActivityThread.java:5343) 
        at android.app.servertransaction.ActivityResultItem.execute(ActivityResultItem.java:54) 
        at android.app.servertransaction.ActivityTransactionItem.execute(ActivityTransactionItem.java:45) 
        at android.app.servertransaction.TransactionExecutor.executeCallbacks(TransactionExecutor.java:135) 
        at android.app.servertransaction.TransactionExecutor.execute(TransactionExecutor.java:95) 
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:2214) 
        at android.os.Handler.dispatchMessage(Handler.java:106) 
        at android.os.Looper.loopOnce(Looper.java:201) 
        at android.os.Looper.loop(Looper.java:288) 
        at android.app.ActivityThread.main(ActivityThread.java:7842) 
        at java.lang.reflect.Method.invoke(Native Method) 
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:548) 
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1003) 
     Caused by: android.os.RemoteException: Remote stack trace:
        at com.android.server.media.projection.MediaProjectionManagerService$MediaProjection.start(MediaProjectionManagerService.java:482)
        at android.media.projection.IMediaProjection$Stub.onTransact(IMediaProjection.java:137)
        at android.os.Binder.execTransactInternal(Binder.java:1179)
        at android.os.Binder.execTransact(Binder.java:1143)

Java sample

is it possible to have a java sample? Thanks

Videoview is empty in screenshot

when i take a screenshot with my screen and it has a videoview which is playing videos, the area of the videoview is completely empty. it shows in black color.

How can I use it in service?

ScreenShotManager requires activity to build. Is there any way to build it without activity? If any code tweak needed then please guide me , I will make it by forking it.

How to get screenshot Object from "it"

I don't know Kotlin very well and in a part of code in the Read Me , it says you can get screenshot object from "it".

val screenshotResult = screenshotManager.makeScreenshot()
val subscription = screenshotResult.observe(
onSuccess = { processScreenshot(it) },
onError = { onMakeScreenshotFailed(it) }
)

How can I get Screenshot object and get bitmap from that ?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.