How to read window content (using accessibilityService) and evoking UI using draw over other app permission in Android?

Question

My last question about the same topic was not clear enough and was put on hold by community and later it was automatically deleted. So, I am explaining that Question in detailed manner so that community can understand and help in a better manner.

I want functionality similar to Voodoo App and MySmartPrice offers.

Now what they do

Step 1: When we open the Voodoo App for the first time they show a small tutorial. At the end of tutorial there is a "Activate Now" button which when pressed take us to Accessibility settings screen.

Step 2: In Accessibility Screen it further guides how to find and unable the Voodoo Service.

Step 3: When we enable it, it further ask to grant "Observe Your Action" and "Retrieve Window Content" permissions.

Step 4: Once we are done granting permissions on accessibility screen, and move to some shopping app or access shopping site via browser and reach product page a voodoo floating button pops up automatically.

Step 5: Once we click on it, it shows the price of same/related/similar products available on other apps or website so that user can check for best deal available.

Reference Screenshot of Voodoo

Now what I want to know form community:

1. How I can show my help UI over Accessibility screen and product detail page.
2. How to detect when a product page is there on screen and to evoke my floating button.
3. How to fetch name of product shown on screen.
4. How to restrict this screen reading functionality for some apps only? (as I don't want to end up in some copyright issue)
5. Is there any tutorial which can help me out? Although I had already tried on Google for any direct tutorial and didn't got any success.

Now why I need this information:

I am planning a app for students which will help them to get there desired (or similar) book (ebook) for free if it is available on my server or on some web location which is available web. I want to restrict it the functionality to some apps only due to copyright issues on some books.

Now what I know from my study regarding this topic (but I am not sure whether I am on right track or not)

• I can take help form Accessibility Service
• Draw Over Other Apps. Example 1 , Example 2

Answer 1

I had tried to summarize how i was able to get thru the problem in a blog post. Anyone who needs some help can take a look into it.

Accessibility Service to the next level

My Previous answer was deleted by moderators may due to i just posted a link to blog. So i am posting my answer again with details here too.

The easiest way is provided by android itself. If you are building something on your own app this is the best and easiest possible way of doing it. You have to use “findAccessibilityNodeInfosByViewId ()” as explained below

@Override
public void onAccessibilityEvent(AccessibilityEvent event) {
    AccessibilityNodeInfo source = event.getSource(); 
    if (source == null) {
        return; 
    } 
    List<AccessibilityNodeInfo> findAccessibilityNodeInfosByViewId = source.findAccessibilityNodeInfosByViewId("YOUR PACKAGE NAME:id/RESOURCE ID FROM WHERE YOU WANT DATA"); 
    if (findAccessibilityNodeInfosByViewId.size() > 0) {
        AccessibilityNodeInfo parent = (AccessibilityNodeInfo) findAccessibilityNodeInfosByViewId.get(0);
        // You can also traverse the list if required data is deep in view hierarchy. 
        String requiredText = parent.getText().toString();
        Log.i("Required Text", requiredText);
    }
}  

If you are building something on other apps and don’t know the res id. You have to build a logic with the combination of parent, child count, level on which your data is found, type of event. You have to look for a pattern in event or source or the combination of above said values to get the required data preciously. Further, you have to setup some tweak in your code to get the accurate data according to your task. Like in our task we have some regex to get precious data.

You have to take care that if the view or UI is changed or res ids are changed your current logic might fail. So, you have to keep a close look at the app on which you are building your service. Following are some example code in which we are getting data without res id that might be useful for you to understand how it will work.

To print source and event

@Override
public void onAccessibilityEvent(AccessibilityEvent event) {
    AccessibilityNodeInfo source = event.getSource();
    if (source == null) {
        return;
    } 
    Log.i("Event", event.toString() + ""); 
    Log.i("Source", source.toString() + "");
}

To get the childcount

source.getChildCount();

If child count is >0 you might have to look into it.

Example Code 1

if (level == 0 && source.getClassName().equals("android.widget.TextView") && source.getText()!=null && !source.getText().toString().isEmpty()){
    // here level is iteration of for loop
    String recivedText = source.getText().toString(); 
    if(source.getClassName().equals("android.widget.TextView") && source.getParent()!=null && source.getParent().getClassName().equals("android.widget.FrameLayout") && source.getParent().getParent()==null){ 
        return recivedText;
    }
}

Example Code 2

if (source.getPackageName().equals("PACKAGE NAME OF APP FOR WHICH YOUR EXPECTING EVENT")) {
    if(event.getEventType()==AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED && "COMPLETE NAME OF ACTIVITY CLASS WITH PACKAGE NAME (if you want it for some specific screen)".equals(event.getClassName())){
        if(source.getText()!=null && source.getClassName().equals("android.widget.TextView") && source.getParent()!=null && source.getParent().getClassName().equals("android.widget.RelativeLayout")&& source.getParent().getParent()!=null && source.getParent().getParent().getClassName().equals("android.widget.ScrollView") && source.getParent().getParent().getParent()!=null && source.getParent().getParent().getParent().getClassName().equals("android.support.v4.view.ViewPager")&& source.getParent().getParent().getParent().getParent()!=null && source.getParent().getParent().getParent().getParent().getClassName().equals("android.widget.FrameLayout")&& source.getParent().getParent().getParent().getParent().getParent()==null){
        return source.getText().toString();
        }
}

It is bit clumsy but to get it working you have to look deep into logs to find a pattern which will take you to your required data.

Update 20/11/2017 Google is suspending all apps using accessibility services for things other than accessibility. I had received a email regarding the same for one of my app and other developers are also receiving same email from google (Reddit). So, I think we should look for some other way of doing things and I request you to please update it here also, if some one comes up with other implementation to achieve same behavior it would help others also.

Thanks Umesh Chauhan