Process scheduling in large display wall environments (Part 2)

This post follows the part1 of the discussion.

What do we need to know to provide fairness in presentation level and achieve better user-perceived performance? We need to know an application’s resource need so that a scheduler can allocate an amount proportional to the application’s need. For now, let’s assume that this information can be found out by the scheduler. Then what else would we need ? If we can define a mechanism to assess a priority value for an application based on users’ interactions it would help achieving better user-perceived performance.

Whereas traditional operating systems employ simple mechanism to assess an application’s priority I present my priority assessment model that can be used to achieve fairness (in presentation level). Below briefly introduce the three components in the model.

Visual Factor

priority layout 0

An intuitive visual factor to determine an application’s importance is the application layout on the display wall as illustrated in Figure 11. The application layout can be categorized into four with two variables (the percentage window overlap and the number of application windows). A fair sharing might be useful in the tiled case (lower right corner) while more system resources need to be allocated to the application whose window is in the foreground in the stacked case in Figure 11. However, prioritizing applications based on their visual factors can be simplified by using a single factor that tells how much of the application window is visible to users.

The visible window size of an application as an indication of user interest is straightforward. Even though an application might not be receiving user input, a large window size can imply high interest. Similarly, if an application’s window covers a significant portion of the display wall, that application is more likely to draw users’ attention. Visible window size is defined as the total size of the visible, non-occluded areas of the application’s window in pixel. This can be easily calculated by subtracting the sizes of portions occluded by other applications.

Interaction factor

Imagine a display wall with tiled application layouts and users are focusing mostly on the applications on the left as depicted in below figure. Or assume two application i and j where Pvisual (i, t) >> Pvisual (j, t) at time t. A user could be interacting more frequently with application j while i’s priority is higher because it has a larger Pvisual value. Typically, user interactions through input devices such as mouse, gyro mouse, touch, gesture, or keyboard indicate user’s interest in an application directly.


Usage factor

Spatial layout and window arrangement patterns are likely to emerge if the display wall is used long enough. To get an insight into display wall’s usage patterns, imagine a virtual grid juxtaposed on the display wall, and use that grid to aggregate priority values of applications in each cell of the grid. The grid can be color-coded by the aggregated priority values forming a heat map; high priority values are indicated as high temperature. For example, in the top layout in the figure below, most of the applications are positioned in the left portions of the wall, causing an increase in the temperature of the left portion of the wall. In the bottom case of the figure, the left side of the wall has higher temperature even though applications are scattered arbitrarily on the wall because users are interacting more with the applications on the left side. In these cases, if a user brings an application window from the right side of the wall (cold region) to the left side of the wall (hot region), then the application will get an immediate priority bonus.