This section will present the Breiny application’s user interface and the available interactions with it, features which will be evaluated both theoretically and by the users. It is composed from a main SurfaceWindow named “Main screen” in figure below, on which all the interaction occurs. Following the requirements for extensibility, modularity and flexibility, the subsequent visible objects were separated into several user controls.
In the following subsections the main screen will be presented first along with its specific interactions, then the discussion will continue with the available item types and their particular available features.
Main screen
The main window acts as an aggregator for all the interactions, providing also support for the other controls and their interactions. It offers access to the main contextual menu, to the tag recognition and visualization, offers the data context through binding against the Session singleton and, because binding is propagated to the children controls, allows subsequent binding context to all of them.
In the figure from the left, the Breiny application was started and a new Brainstorming session is created. In this specific case the Brainstorming Items Repository didn’t already contain Brainstorming Items, thus none of these appeared on the application screen. But the users may have prior access to other devices that can interface with this thesis proposed system (iPhone, iPad or Adobe Air client) as mentioned in Architecture chapter and send they initial suggestions. If so, their Brainstorming items will show on the newly created Brainstorming Session.
Being a global User Interface element, the contextual menu (named ElementMenu in Microsoft Surface SDK) already mentioned in the previous paragraph is activated by tapping on the Microsoft Surface’s Screen. If the user taps again on the screen, this time in the glowing circle that appeared (where the red finger icon is shown), a contextual menu with following actions will appear:
- New Brainstorming Item – creates an empty Brainstorming item and adds it to collection.
- Clear Tweets – clears all the items from the TweetsList from the Session object, triggering the delete all visible items of corresponding User Control from the main screen. This feature is used when all needed Tweets were used and the Brainstorming Session users need more space on the screen.
- Clear Flickr images – same behavior as for Clear Tweets, but with the mention that the deleted items are from Flickr.
- Print Screen – this feature allows quickly saving a picture of what is currently on the main screen.
Tag interaction
One of the features implemented by Microsoft Surface SDK is the possibility to recognize objects that make contact with the screen specific shapes and assign them to data structures. The Tag system is used to quickly identify already set associations between recognized Tag’s number and a data structure stored in the Operating System Registry. The data stored was linked with the pre-configured users which will perform the role of Session Master. The data structure contains among others the Name, the account on BSCW server, the account used for Brainstorming Item Repository.
As future work, the system can be extended and the tags printed also on the Research Institute employees’ badges, such that the interaction with the Brainstorming system proposed by this system to be largely improved.
When a preconfigured Tag is placed on the screen, the visualization from below figure is shown, prompting the user to insert its password. Since the server is already set and the username is retrieved from the Registry, this is the single information required for a user to be able to access the panel presented in below- right side figure, allowing him or her to perform more actions.
User Controls
The custom User Controls can be found in Breiny’s class diagram in the View – UserControls section. The name UserControls is used in the context of the Visual Studio 2010 development environment and represent modular code encapsulated into easily reusable controls. They are still part of the View paradigm in MVVM pattern along with the already presented Main SurfaceWindow. Worth to mention is that they are not consisting only from visual elements, but rather composed from both UI elements and their attached code behind. In order to respect the separation in the MVVM pattern, only the required code was implemented in the code behind class, the rest of the logic being implemented either in helper classes or in the Session object, which represents the ModelView paradigm in MVVM pattern.
Each UserControl is placed inside of the Main SurfaceWindow as a children control and enables a key scenario in the Breiny’s lifetime:
- ucSVBrainstorming and its children’s template ucSVItemDefault provide the visual interface and interaction with the most important items in the prototype: the Brainstorming Items (for all the children templates the interactions and visuals will be presented in the next section – Items).
- ucSVFlickr and its children’s ucSVItemFlickr provide visual interface and interaction for the system mash-up with the Flickr website, allowing users to select, preview and manipulate them;
- ucSVTwitter and its children’s ucSVItemTwitter, as the above Flickr counterpart, provide visual interface and interaction for the system mash-up with the Twitter website, allowing users to select, preview and manipulate them;
- ucSplash is used as a splash screen, shown at the application start;
- ucBrainstTitle was inspired from the Berufsakademie-Ravensburg application (Dräger, et al. 2008): the revolving text written along a spiral will allow the Session goal to be clearly specified.
User Controls – Items
The items represent the visualization of the data items – Brainstorming Item, Flickr and Twitter item. They have associated menus for special actions that can be applied to them. This subsection will discuss in detail the role each has, how can each item be created, how purpose it server, what actions or features it offers and its lifetime.
The Brainstorming Item
It is the most important item available, as it represents the center of the Brainstorming interaction – it contains the smallest unit of a Brainstorming session. Because of this, it is the most complex object in the prototype and enables a set of key scenarios.
In figure from the left is presented the User Interface of Brainstorming item. By taping on it, the contextual menu appears and the user can access the object’s features:
- Edit – after pressing it (Figure 28), a set of controls are made available to edit the content of the item. Pressing again on Edit will toggle this mode off, hiding the controls. The active mode is marked with the yellow color (in this case the mode is Drawing). The three editing modes are:
- software keyboard on the screen,
- inserting the text via Microphone
- draw on the screen with fingers or Infrared pens. By clicking on the colored dot, the users can choose another color and pressing the eraser button will switch between drawing and erasing.
- Group – after activating this mode (Figure 29), another set of controls appear, allowing the users to change the background color of the item and thus mark it visually as belonging to a color category. Pressing again on it will toggle this mode off, hiding the controls.
 Figure 28 – Edit menu with painting activated |
 Figure 29 – Grouping menu, now item belonging to another color coded category |
- Relating allows creation of a relational structure, linking related items through lines. For this, first it must be activated and afterwards, one can drag and drop the activated Brainstorming item over another Brainstorming Item, which will result in creation of a white line linking them and representing their relation (Figure 30). To delete this relation, the user must tap once the line, transforming it into a dashed red line (Figure 31) and then tap again the line to finally delete the relation between items (Figure 32). If the second tap is not executed in 3 seconds, the line will revert to its original state, white without dashes.
 Figure 30 – Related Brainstorming Items |
 Figure 31 – Tapping once on the relation will mark it for deletion |
 Figure 32 – Second tap on the relation deletes it |
- Send To – when user activates this submenu, the prototype retrieves from Brainstorming Items Repository the list of all active users and adds them as children to the submenu. When one of the children is selected, the Brainstorming Item is deleted from the prototype list and sent back to the repository, this time marked to be read by the target user.
- Send back – it sends back a Brainstorming item to the user that originally created it. This feature is available when the prototype received this item from another source, e.g. from an iPhone user.
The Twitter / Flickr Item
The paper proposed itself to study the importance of using data from additional social networks, Twitter and Flickr, in order to improve the usability of the prototype. The question raised is if would it be useful to query these services when the users lack ideas or they face a blocking situation in which they require some fresh perspective on their goal. Because these two item types are closely related, this section will present them both, underlying also the differences that exist between them.
First and foremost, the data retrieval can be activated by a Session Master, after its login into the prototype using the already provided tag. Once logged in, on new tag activation, user can insert in the tag visualization window (Figure 26) a word into the textbox and press either the button with Twitter image in order to retrieve Twitter public timeline or on the button with the Flickr logo to access Flickr public images. Once retrieved, the data will be shown as in the Figures 33 and 34.
 Figure 33 – the Twitter Item and its contextual menu |
 Figure 34 – the Flickr Item and its contextual menu |
In Twitter Item, the background is the Twitter bird, and the content is split in three areas: upper left can be seen the picture of the person who created the tweet, in upper right his name and his Twitter id. Underneath is the text of the tweet.
In case of Flickr item, the background is the actual image uploaded by someone and one can identify the same three areas, upper left being the Flickr icon, upper right the picture file name, and in the underneath the picture description.
These items are temporary items; a new search on Twitter or Flickr will delete the old items and replace them with new ones. This decision was made because the screen would get very crowded with all these items, which are not always required after the first view. The mechanism to make them persistent is available as a contextual menu which allows Brainstorming Users to “promote” them to full Brainstorming Items. This way the users can also edit, group and relate them.
When all Twitter and Flickr items which contain valuable information were transformed into Brainstorming items, users may choose to remove the unused ones by tapping on the screen and choose from the contextual menu “Clear Tweets” or “Clear Flickr images”, recovering valuable screen space for further actions.
