NanoOS™ and Application Programming Interface (API) Sony defines a new phase in highly integrated audio, video and graphics LSI systems As digital video becomes a reality in many applications such as digital versatile disc (DVD), digital video broadcasting (DVB), digital television (DTV) and digital broadcasting satellites (DBS), a significant number of application specific integrated circuits (ASIC) have been introduced in the last decade. Most of these ASIC solutions centered around supporting the widely supported digital video standard MPEG-2. MPEG-2 not only provides higher quality video compression when compared to other standards such as MPEG-1, but also has made its way into system standards for DVD, and the ATSC standard for digital television in the U.S. These standards are widely backed by their corresponding industry leading companies such as movie studios and digital broadcast companies, respectively. The first attempts -- which date back to 1993-1994 time period -- were made to satisfy the needs of the digital video market and were immature and fairly simple in terms of features compared with today's technology. During this "initial phase", the main focus of semiconductor companies was time-to-market in order to grab a considerable share of the market and build the expertise in the new field of digital video quickly, by trying to satisfy an embryonic set of emerging markets. The example integrated circuits (IC) of this era featured separate ASICs for audio and video processing, and relied on full low-level software control from a host processor (this is referred to as "Micro" -- short for Microprocessor -- by some industry members). These ASICs served as trial platforms for many companies. All of the ASICs in this phase relied on standard audio/video synchronization and simple error concealment techniques all to be performed by the host processor. In the following two years, some components were introduced with more functionality such as integrated microcontroller, partly integrated audio/video decoding. These devices still required partial software control from the host processor, and could only support primitive levels of features. This phase, which is referred to as the "low integration phase" was a result of the growing marketplace for compression related products and a sign of the end of the trial phase for many semiconductor companies. In the last year, the situation has changed dramatically with the adoption of the DVD standard and the wide acceptance of the DVB standard. Both standards are a result of intense research and development for many years and include many features which are formed by the heavy negotiations among industry leaders such as Sony. Meanwhile as the market became more mature, a magnitude of applications grew significantly and highly integrated devices started appearing in key semiconductor company road maps. However, to date all of the devices clearly give away the strategy of trying to squeeze in as much functionality as possible in the same silicon for cost savings. The devices proposed by most companies lacked the re-thinking at the system level as most of them were achieved by brute-force integration of pre-developed devices. Sony Semiconductor Company of America (SSA) has defined a new phase in highly versatile audio, video and graphics LSI systems. In the new approach, SSA has developed a chip based on a hardware-software environment in which a significant portion of the functionality is carried out in software on-chip by the way of highly programmable hardware cores. SSA's first attempt at this approach is demonstrated on the CXD1930Q DVD/VCD/DVB chip, which forms as the basis for its Virtuoso™ family of digital media devices. SSA's highly integrated software brings the level of functionality that used to be performed traditionally on the host processor into the chip. SSA has developed an advanced, industry-wide open API, enabling enhanced functional customization options to set/system developers. SSA's API set is multi-level and portable across different platforms (Figure 1). In addition to the multi-level API, SSA's system software integrates an advanced set of features, some of which are an industry first. SSA's software also enables smooth transition to future products such as HDTV through its portable architecture.   Figure 1. Sony Virtuoso Software Architecture The Virtuoso family of products come equipped with SSA's embedded media software, providing not only real-time control for audio, video and graphics, but an advanced feature set as well The need to handle different applications requirements for different markets like DVD and set-top box technology, led SSA to think about optimizing the traditional system partitioning problem in hardware and software without sacrificing the need to re-design the underlying hardware architecture. SSA's API software in conjunction with the concept of a small nano-kernel running on a target Virtuoso family chip led to the concept of the NanoOS kernel architecture. The small compact kernel solves all of the traditional embedded system hardware/software tradeoff issues that have to be considered by the system architect for a particular application. The key tradeoff issues are shown in Table 1. A.   Handle both real-time and non-real-time tasks effectively and efficiently B.   Have more features, yet keep the cost down C.   Provide product differentiation through flexible architecture D.   Customizability E.   Smooth transition for customers to Virtuoso family of products F.   Provide portable software across different customer platforms Table 1. Key Trade Off Issues The Virtuoso Media Software API This nano-kernel running on Sony's Virtuoso family of semiconductor digital media solutions is the glue to a diverse set of API's developed by SSA for different applications and is called NanoOS. The feature set flexibility enabled by the NanoOS API set for a particular application without a change in the underlying hardware is the key to the scalability and portability of the system architecture. The Virtuoso media software supports an industry-wide open API set for accessing its powerful set of features. The API is multi-level, thus enabling different levels of control depending on the host processor software architecture. The API set for DVD includes advanced trick play for both audio and video trick play, sub-picture graphics control, on-screen-display graphics control, still image tiling and picture-in-picture, angle picture viewing and graphics and image scrolling and programmable filtering with optimum filter selection based on arbitrary sampling ratio inputs that enable features such as zoom-in and zoom-out. The API set includes more than 100 API's and is built on a hierarchical structure, enabling easy customization, product differentiation not only for SSA but also for any customer programming to the SSA API. Through its multi-level API, SSA carries most of the functionality traditionally provided at the host processor level to on-chip. Examples of DVD APIs are shown in Table 2. FlushCodeBuffer(bufferID) SetTVSystem(tvSystem) SetDiscMode(discMode) SetDiscMode(discMode) SetAudioCodec(codingType) SetAudioStreamId(audioStreamId) SetSPStreamId(SPStreamId) SetArea(startAddress,endAddress) SetVideoStreamId PlayReady() Play() Pause() SlowF(speed) I_FF() IPP_FF() StepF() I_FR() IPP_FR() SlowR() StepR EnableSubPicture() DisableSubPicture() SetHilight(Btn_gp,Btn_Number,Color) ResetHilight() MoveSP(SPOffset) Fade(frameCount,color) SearchStartCode(startCode) SearchPictureType(pictureType) SearchPictureStructure(pictureStructure) SearchReferencePicture() SearchPictureTimeCode(timeCode) DecodeNextPicture() SkipDecodingNextPicture() DecodeVarSpeed(speed) PlayVideo() StopVideoDecode() PauseVideoDecode(frameCount) DecodeVideoBuffer(bufferPointer) DecodeNextIPFrame() DecodeNextIFrame SearchTemporalReference(temporalReference) SetDefaultQMatrix() SetDefaultSeqHdr() InsertSeqHdr(sequenceHeader) SearchVideoPTS(targetPTS) GoToVideoTarget(Target,TargetType) SearchNextSyncWord() PlayAudio() DecodeNextFrame() SkipDecodingNextFrame() PauseAudioDecode() StopAudioDecode() SearchAudioPTS(audioPTS) SetDynamicRange(dynamicRange) MuteAudio() UnMuteAudio() DisplayVideoField(bufferPointer) DisplayVideoFrame(bufferPointer) RepeatVideoDisplayField(buffer Pointer,fieldCount) RepeatVideoDisplayVideoFrame(bufferPointer,frameCount) SetChromaInterpolationMode(CIMode) SetChromaFilter(chromaFilterTaps[]) SetLumaFilter(lumaFilterTaps[]) SetHorizontalResampler(resamplerTaps[]) Table 2. DVD API Examples Sony's Virtuoso media software provides many industry first features in its application category The following list is an example snapshot of the features that are possible as part of the NanoOS API set supported by SSA's Virtuoso media software for DVD and VideoCD applications (Table 3). MPEG1 and MPEG2 video control MPEG1, MPEG2, LPCM and AC-3 audio control DVD and VideoCD (VCD) decoding control DBV decoding control NTSC and PAL decoding control Letter-box and pan-scan aspect ratio conversion Down-sampling and Up-sampling with anti-alias filtering for arbitrary ratios (horizontal and vertical) Proprietary high quality vertical and horizontal filters with automatic selection by the algorithm Angle Picture decoding and display User definable OSD header formats parsed and controlled by on-chip software 2.53 frame decoding for optimum memory usage Proprietary Audio/Video/SubPicture synchronization algorithm with optimum quality and memory management Real-time operating system (NanoOS) Advanced DVD and VCD trick play commands Advanced DVD, VCD and DVB API (more than 100 commands) Automatic high quality 3:2 pull-down for both MPEG1 and MPEG2 Advanced proprietary error concealment PAL decoding management in 16Mbits SDRAM Closed caption decoding and control NTSC/PAL (TV system) conversion (bitstream to display) High wuality chroma interpolation with automatic selection from multi filter banks TV system conversion with audio/video synchronization SubPicture decoding and display Asynchronous decoding control (non VSYNC based) for fast trick play DVD Navigation support by PCI/DSI information decoding and control E-STD model support Treatment of video gap P-STD model support Table 3. NanoOs API Set Features SSA's NanoOS, industry first in its application category, provides the optimum trade-off in resolving the real-time, non-real-time conflict Along with its new Virtuoso media software, SSA's NanoOS was created as previously explained, to handle real-time and non-real-time tasks in supporting the advanced feature set. NanoOS holds 256 bytes in local instruction memory and enables, fixed priority task scheduling for both interrupt domain (real-time) and non-interrupt domain (non-real-time) tasks. The task scheduling is optimized for both performance and instruction space. NanoOS comes equipped with a virtual memory management machine within the 256 bytes. The virtual memory manager supports programmable memory size and location, swap prediction, multiple entry function support, variable size swap in/out and parameter passing at the OS level with context switching. NanoOS, provides a compact kernel for handling real-time tasks and serves as a layer between the application and the hardware for several reasons such as, isolating real-time and non-real-time tasks with a clear boundary, portability and ease of programming. Thus, SSA's NanoOS, an industry first in its application category, provides the optimum trade-off in resolving the real-time, non-real-time conflict. NanoOS NanoOS is a real-time embedded operation system first used in SSA's CXD1930Q. It has major features that highlight the simple structure and high performance of the CXD1930Q. In order to explain its structure, both the software architecture and the CPU instruction memory (IMEM) structure of CXD1930Q are explained below. CXD1930Q has 2K (each with 16-bit width) instruction memory. Therefore, the on-chip software is divided into "resident" and the "non-resident" parts. It is easy to see that the former is always allocated in the CPU IMEM and the latter is in DRAM and will be swapped into CPU IMEM when needed. Also, the non-resident routine will be overwritten when the next one has been swapped. The CPU IMEM can be partitioned into roughly 4 areas: (1) Interrupt vector table; (2) NanoOS (3) Resident routines, including interrupt service routines (ISR) and other time concern routines; and (4) Swapping area, for swapped non-resident routines. NanoOS consists of two major parts: one is "task-manager", and the other one is "swap-manager". The former controls all main tasks and decide which task is going to be executed by checking the priority-register bit by bit, then task-manager passes the task information to swap-manager which swaps the right task, after pushing and protecting the necessary data. Swap-manager takes care of virtual memory management process and the return path when the task routine is finished. Besides the above processes, NanoOS also allows non-ISR routines to call other routines, including the resident to the resident, the resident to the non-resident, the non-resident to the resident, the non-resident to the non-resident, and any type of nested calls. For this part, all calls are sent to the swap-manager directly without going through task-manager. As an example, CXD1930Q software has 16 interrupt service routines (ISR), 32 main scheduled tasks, and other functional routines. The interface between two routines is minimized, which enables object oriented software development. It also avoids any interference from other routines. The NanoOS software architecture can be seen in Figure 2. Other objects such as interrupt service routines, main scheduled tasks, functional subroutines are based on NanoOS. There are two ways to communicate between two routines. One way is to use the "SCHEDULE" ("ISCHEDULE" in ISR) macro to schedule one task so that task-manager can execute one by one according to their priorities. The other way is to use "CALLS" macro through swap-manager, which serves function calls by swapping the routine from DRAM into CPU IMEM if it is non-resident or just jumping to the right location in IMEM if it is resident. Swap-manager also supports the nested calls, which means the sub-routine being called can still call other routines before it returns. So, from a software programming point of view, the macro "CALLS" is fully transparent and no push-and-pop procedures are needed.   Figure 2. Software Architecture of NanoOS For performance reasons NanoOS supports an important feature. If there are two small functions associated with each other, they are packed into one section so that both of them are swapped in the same time if any one is called. When one routine needs to call both of them or one of the sub-routines needs to call the other, the swap-manager only swaps this section once by checking out both sub-routines that have the same section-ID and have already been swapped; i.e., it is not necessary to swap the same section again. Because such situations happen often, this feature is very useful and the system performance is increased dramatically since the swapping frequency is reduced. Summary Sony Semiconductor of America (SSA) through its API Set supporting its NanoOS software on-chip technology is enabling system developers and OEMs to design applications for a wide spectrum of multimedia applications. The NanoOS and its API set will allow OEMs to offer advanced features to be integrated easily and cost effectively while being able to support a diverse set of application requirements with using the Virtuoso family of semiconductor devices from SSA. Home | Products | Sony Semiconductor | Sony Spotlight What's New | Customer Support | Search Buy It! | Career Opps | Feedback | Resource Library PR Archive | Privacy Policy | Site Map Send email to the Semiconductor Solutions Division © Copyright 1996-2008 Sony Electronics Inc. All Rights Reserved. Legal/Trademark For information or questions concerning consumer products such as digital cameras, DSS, televisions, VCRs and more, please visit Consumer Customer Service or call 1/800/222-7669.