Detailed software design
Detailed software design	1
Diagram	2
General	3
Approach	3
Score	3
Cast	3
Files and folders	3
Specification	4
1.	Input Processing	4
2.	Editron Table	4
3.	Capsules	5
4.	Capsule Tracker	6
5.	Processing and output to KeyWorX	7
Database Architecture	8
Table	8
Database Sample (colours don’t mean anything)	9

Diagram

General
Approach
The original ToolWorx system was designed as a single frame object oriented and entirely pupetted  system. Each module was to be created by a parent script and everything should have been kept in a single global tree. The system was also designed so that it could easily be ported from a virtual demo system to an actual tangible system with little need for modification. This hasn’t quite worked out.
Two main obstacles appeared:
First, we were unable to establish a streaming XML communication line between KeyWorx and Director. This resulted in our inability to use 4 controllers and what’s worse, in our inability to have Director send its output for KeyWorx to render. See the notes on implementation in the specification below for further details on the solutions employed. Generally, this required a more score-based approach, and virtually no puppeting.
Second, the capsule tracker, which is the component that was to use the data from either a tagging system on tangible objects or their virtual equivalent, was outsourced to a 3rd year student. He didn’t deliver it, not on time and not after he was given several extensions. The patch for it is probably not as easily scalable to the tangible system as could be hoped. See the relevant note on implementation for further details.
Score
Originally intended as a single frame movie, it has become apparent that performance issues require a less orthodox approach for this application. Therefore, some initialization processes are located in the “on exitFrame” event of frame 10, while the actual operation loops continuously on frame 15. FPS rate is set at 15, although actual performance is usually as low as 2-3 FPS. Place-holder sprites are taken over dynamically on run-time, and can optionally be entirely puppetted, should the system be controlled by XML events from keyworx. See code comments for specific details.

Cast
The internal cast includes all scripts and behaviours, as well as the flash object of the Editron table and the collision detection map. All other graphics for Cursors, segments, capsule shapes etc. are linked.

Files and folders
Application - Root:toolworx1.2.app
Media Database - Root:CapsulesDB.txt
Graphic files (thumbnails and icons) - Root:[]
Media Files: Root: Files: Media

Specification
1.	Input Processing (Noam)

1.1.	XML Input from KeyWorX
1.1.1.	XML socket listens and captures incoming XML strings
1.1.2.	XMLparser turns XML into lists
1.1.3.	An event handler turns these lists into cursor states
1.2.	 Four 'cursors' corresponding with 4 controllers
1.2.1.	 ID (FGColor, Visualy corresponding with physical gamepads)
1.2.2.	X and Y position (locH, LocV)
1.2.3.	 Status (Variable, memberName)
1.2.3.1.	mouseUp
1.2.3.2.	mouseOver (when over a draggable capsule)
1.2.3.3.	onDrag (when dragging a capsule)
(note on implementation: for reasons which are still under investigation, the XML communication with KeyWorx, using a Flash XMLSocekt object, could not be established. Therefore, two alternative systems have been implemented, whereby either the system is controlled by a mouse – in which case it becomes a single user application – or it generates its own events at random and then handles them “as if” they were XML messages from keyWorx.
2.	Editron Table (Thomas De Bruin)

2.1. Tracks 
2.1.1.	 2 Audio rings (blue)
2.1.2.	 2 Image rings (red)
2.1.3.	Segments (when capsule is over an appropriate track)
2.1.3.1.	Duration lights up on track
2.1.3.1.1.	Vector graphics based on capsuleDB data on media file length
2.1.3.1.2.	At later state - out point can be determined and thus segment length may change.
2.1.3.2.	Clip text appears
2.1.3.2.1.	Retrieved from capsuleDB
2.1.3.3.	Thumbnail or icon appears
2.1.3.3.1.	Pathname to file retrieved from capsuleDB
2.1.3.3.2.	File retrieved from folder (see files and folder section below)
2.2	 Moving play head (beam of light, should also be visible through the objects on table)
2.3. Storage
2.3.1.	Initially the capsules are stored on the outer track of the Editron table
2.3.2.	All the capsules in the storage have a fixed default location, determined by an ID corresponding to the capsule ID from the capsuleDB
2.3.2.1.	When a capsule is dropped besides the tracks it goes back to its default place in storage (based on state returned from capsule tracker).
2.3.3.	All possible media is visible through icons that are layered on top of the capsules.
2.3.3.1.	15 pre-recorded audio capsules, split into 3 different types: spoken word, music & sfx
2.3.3.2.	5 pre-recorded moving image capsules
2.3.3.3.	5 pre-recorded still image capsules
2.3.3.4.	1 live audio-capsule
2.3.3.5.	4 live moving image capsule
2.3.3.6.	1 live text capsule
2.3.3.7.	1 live drawing capsule

(note on implementation: the Editron table was implemented as a flash object. Its methods for creating the tracks and storage, the play head and the segments are called by Director where appropriate. Director sets the time, number of tracks, location of segments, description text and so on, while the flash object draws them, and in the case of thumbnails also loads external graphics).

3.	Capsules (Noam)

In the virtual system, capsules are generated according to the database. A handler reads the database, creates the required capsules and positions them in assigned default locations on the Editron table. Then, the user can drag them to other locations. The capsule tracker determines where the capsules are and transmits that back to the capsules and the Editron table, which negotiate and determine the appropriate states and behaviours.
Each capsule is comprised of 4 sprites: a bitmap, an underlying vectorshape to enable collision detection (also responsible for mouse detection in case of a mouse controlled implementation), an icon, and descriptive text (not implemented).
3.1.	Tag (unique ID from database)
3.2.	Model (media capsules only at this stage - assembled by data from DB)
3.2.1.	 Shape (live, pre-recorded, audio, video)
3.2.2.	 Colour (audio, video)
3.2.3.	Icon (one of a possible list…[complete])
3.2.4.	Physicality
3.2.4.1.	Collision detection - capsules cannot overlap
3.2.4.2.	Dragable
3.2.4.2.1.	onMouseDown an objects is dragged
3.2.4.2.2.	onMouseUp an  Object drops and is  not dragged anymore
3.3.	Capsule States (determined by data from the location tracker)
3.3.1.	Inactive (in storage)
3.3.2.	Hover above capsule in storage
3.3.2.1.	Capsule lights up
3.3.3.	Dragged
3.3.3.1.	Object follows cursor
3.3.3.2.	Cursor appearance changes
3.3.4.	Connected right (on a track)
3.3.4.1.	Capsule lights up?
3.3.5.	Connected wrong (on a track)
3.3.5.1.	Capsule does not light up?
(note on implementation: collision detection between capsules was not implemented, as it would have required too much processing power, which would have slowed performance below any minimal acceptable threshold)
4.	Capsule Tracker (Ton)

4.1.	Trigger: On every frame
4.2.	Internal data: 
4.2.1.	Knows where all the Editron elements are (static data)
4.2.2.	Knows which capsules can be placed on which track (glean from capsuleDB[i].ID.capsuleType.media.mediaType, where audio capsules fit on audio tracks and image capsules on video tracks)
4.3.	Basic process: For every capsule, identified by tag info in capsuleDB, return:
4.3.1.	locH
4.3.2.	locV
4.4.	Further processing: Based on 4.2 & 4.3 and when a capsule is not dragged (see 1.2.3 above), determine:
4.4.1.	Whether it is in storage or on the table.
4.4.2.	If in storage, whether in default location or not (see 2.3.2 above)
4.4.2.1.	Return state
4.4.3.	If on table, whether on appropriate track or not
4.4.3.1.	Return state
4.5.	When a capsule is dragged 
4.5.1.	if on table
4.5.1.1.	if on appropriate track
4.5.1.1.1.	Return state
(note on implementation: The student commissioned with this module did not deliver it. Therefore, it’s critical functions have been assigned to the cursor and capsule scripts. This does not guarantee good portability to a tangible system. On the other hand, performance is better this way. 4.1 has been eliminated, and the tracking is triggered based on capsule and mouse events. 4.2-4.4 are handled by the capsule script, which compares a capsule’s drop point to a collision map and decides on correctness and so on. 4.5 has been eliminated.)
5.	Processing and output to KeyWorX (Noam)

Keyworx has no timeline so we can’t place different media files one after another in time on a layer without loading and unloading the files on-time. Therefore, each media layer in Director can use 2 media layers in Keyworx. This minimizes the delay in transition from one media-file to another. So Director distributes the loading and unloading from media on an Editron track over 2 keyworx layers. Besides preparing the media layers, filters and transitions that are applied to a media capsule need to be prepared in keyworx., so the connected filter or transition needs to  be connected to the right media layer.

When the placed media needs to be played or stopped, Director sends a command to Keyworx to play the right media layer.

5.1.	Director Xml output
5.1.1.	Dynamically sort the media content on one Editron media layer into two media layers in keyworx .
5.1.1.1.	Load media in the target layer
5.1.1.2.	Place media capsule (for layer reservation)
5.1.1.3.	Control media layers in Keyworx
5.1.1.3.1.	Play a media layer
5.1.1.3.2.	Stop a Media layer
5.1.1.3.3.	
(note on implementation: due to the XML communication problems, the absence of a capsule tracker and the resulting time constraints, this module was not implemented at all)

Database Architecture
The media capsules DB is implemented as a static ASCII text file, with a single Director formatted property list with the following structure:
Table[2nd priority items]

Property	Data type	Values/description	
1	2	3			
[[1],[1],[1], … [1]] 	List	An array of property lists, each one describes a single capsule	
	#tag:  	string	Capsule’s unique name	
	#capsuleType 	string	Media, filter or transition capsule	
	“media”	Case		
	#mediaType  	String	PreRecorded[/live] - Image/audio	
	“preRecordedImage”	Case		
	#subtype	Integer	1=Video, 2=still	
	#name	String	File name	
	#description	String	Free text describing content	
	#thumbnail	String	File name	
	#duration	Integer	Time in seconds	
	“preRecordedAudio” 	Case		
	#subtype	Integer	3=Music, 4=Spoken, 5=FX	
	#name	String	File name	
	#description	String	Free text describing content	
	#thumbnail	String	File name	
	#duration	Integer	Time in seconds	
	“liveImage” 	Case		
	#subtype	Integer.	6=Image, 7=text, 8=draw	
	#name	String	File name	
	#description	String	Free text describing content	
	#thumbnail	String	File name	
	#duration	Integer	Time in seconds	
	“liveAudio” 	Case		
	#subtype	Integer	9=Mic	
	#name	String	File name	
	#description	String	Free text describing content	
	#thumbnail	String	File name	
	#duration	Integer	Time in seconds	
	“filter”  	Case		
	#name	String	File name	
	#thumbnail	String	File name	
	#level	Variable int.	1-100, default=50	
	“transition“ 	case		
	#name	String	File name	
	#thumbnail	String	File name	
	#duration	Variable int.	0<->30, Default=2	

Database Sample (colours don’t mean anything)
[[#tag: "1", #capsuleType: "Media", #mediaType: "preRecordedImage", #subType: 1, #name: "Video1", #description: "This is video 1", #thumbnail: ":Files:Graphics:video1.pict", #Duration: 15],[#tag: "2", #capsuleType: "Media", #mediaType: "preRecordedImage", #subType: 2, #name: "Still1", #description: "This is Still1", #thumbnail: ":Files:Graphics:still1.pict", #Duration:5],[#tag: "3", #capsuleType: "Media", #mediaType: "preRecordedAudio", #subType: 3, #name: "music1", #description: "This is music1", #thumbnail: ":Files:Graphics:music.psd", #duration: 23], [#tag: "4", #capsuleType: "Media", #mediaType: "preRecordedAudio", #subType: 4, #name: "spoken1", #description: "This is spoken word", #thumbnail: ":Files:Graphics:spoken.psd", #duration:7], [#tag: “5”, #capsuleType: “Media”, #mediaType: “preRecordedAudio”, #subtype: 5, #name: “Plink!”, #description: “Sound effect”, #thumbnail: “:FilesGraphics:SFX.psd”, #duration: 2]]