22
CPPM, IN2P3-CNRS et Université d'Aix-Marseille II Module Management Controller mezzanine board - Specification Issue ......................................... : 3 Revision .................................... : 0 Reference ................................. : Created ..................................... : 15 December 2009 Last modified ............................ : 31 March 2011 (Jean-Pierre Cachemiche) Prepared by : J.P. Cachemiche Centre de Physique des Particules de Marseille (CPPM)

Module Management Controller mezzanine board Specification · Module Management Controller mezzanine board -Specification ... In this note we describe the specification of a mezzanine

  • Upload
    hakhanh

  • View
    223

  • Download
    0

Embed Size (px)

Citation preview

CPPM, IN2P3-CNRS et Université d'Aix-Marseille II

Module Management Controller mezzanine board

-Specification

Issue ......................................... : 3Revision .................................... : 0Reference ................................. : Created ..................................... : 15 December 2009Last modified ............................ : 31 March 2011 (Jean-Pierre Cachemiche)

Prepared by : J.P. CachemicheCentre de Physique des Particules de Marseille (CPPM)

MMC mezzanine board specifications, J.P.Cachemiche Ref.: Issue 3; Revision 0

Date: 31 March 2011

Abstract

In this note we describe the specification of a mezzanine card containing the Module Management Controller (MMC) needed to control an AMC board plugged in a µTCA crate.

Document Status Sheet

1. Document title: MMC mezzanine board specifications

2. Document Reference Number:

3. Issue 4. Revision 5. Date 6. Reason for change

1 0 25 November 2009 First draft

2 0 26 October 2010 Modified initial connector by a keyed one with mechanical guidance

3 0 11 March 2011 Changed connector again for one with a smaller footprint.Definition of pins reserved for RTM management.

2/22

MMC mezzanine board specifications, J.P.Cachemiche Ref.: Issue 3; Revision 0

Date: 31 March 2011

Table of Contents

1. Introduction ......................................................................................................4

2. Role of the MMC .............................................................................................. 5

3. MMC hardware implementation .....................................................................7

3.1. Mechanics ..........................................................................................................73.2. Functionalities ..................................................................................................93.3. Interface with the board ...............................................................................10

4. MMC software implementation .................................................................... 12

4.1. Core code .........................................................................................................124.1.1 Led management ...................................................................................124.1.2 Payload management ...........................................................................134.1.3 Sensors management ............................................................................144.1.4 FRU information ....................................................................................144.1.5 IPMI commands .....................................................................................18

4.2. User code .........................................................................................................18

Bibliography .......................................................................................................19

List of Figures ....................................................................................................20

List of Tables ......................................................................................................21

3/22

MMC mezzanine board specifications, J.P.Cachemiche Ref.: Issue 3; Revision 0

Introduction Date: 31 March 2011

1. Introduction

This document describes the features of a Module Management Controller (MMC) compliant with the AMC specification [1]. This module is implemented as a small mezzanine card sufficiently small to be mounted on any kind of AMC module.

The hardware of this board as well as the firmware is a joint develoment of DESY, CPPM and CERN. Initial code was given by courtesy of Kay Rehlich and Vahan Petrosyan from DESY.

The board has been tested so far in µTCA crates. This is why the description below refer mainly to µTCA environment. Operation on ATCA carrier should however be possible.

4/22

MMC mezzanine board specifications, J.P.Cachemiche Ref.: Issue 3; Revision 0

Role of the MMC Date: 31 March 2011

2. Role of the MMC

The typical implementation of a MMC in a µTCA crate is represented in Figure 1.

In a µTCA system, Hardware Platform management consists in monitoring system information and controlling the system appropriately. For this a Shelf manager usually located on the MCH board communicates over an Intelligent Platform Management Bus (IPMB) with MMCs located on the AMC boards.

The MMC role is to:

• monitor functions (board temperature, component temperature, voltage levels, fan speed, ...)

• monitor states (hot swap, cpu error, power good, ...)• provide the board asset information (manufacturer, product name and model

number, serial number, geographical information, version, …)

With this information, the Shelf Manager is able to:

5/22

Figure 1: Hardware Platform Management in a µTCA system

MMC mezzanine board specifications, J.P.Cachemiche Ref.: Issue 3; Revision 0

Role of the MMC Date: 31 March 2011

• determine if the shelf can supply the necessary power to a module,• activate or deactivate it,• manage the sequence of payload board power-up,• manage the removal of an AMC board from a platform and insertion of a new one

while the power is still on and the system is still operating,• dynamically modify the fan speed based on temperature to keep the system at a

constant cooling state.

Exchanges between the MCH Carrier Module Controller (MCMC) and the MMCs comply to the Intelligent Platform Management Interface (IPMI) protocol. [2][3][4]

6/22

MMC mezzanine board specifications, J.P.Cachemiche Ref.: Issue 3; Revision 0

MMC hardware implementation Date: 31 March 2011

3. MMC hardware implementation

3.1. Mechanics

It is implemented as a mezzanine board interfaced with the AMC board through a dedicated 40 pins connectors.

The connector on the AMC board is a female Samtec SS4-20-3.0-L-D-L-K-TR as shown in Figure 2.

The mating connector on the MMC mezzanine board is male Samtec ST4-20-1.0-L-D-L-P-TR of the same manufacturer as shown in Figure 3.

The dimensions of the MMC board are 39 x 20 mm. The position of the board relatively to the AMC board is shown in Figure 4.

7/22

Figure 3: MMC connector

Figure 2: AMC board connector

MMC mezzanine board specifications, J.P.Cachemiche Ref.: Issue 3; Revision 0

MMC hardware implementation Date: 31 March 2011

The 40 pins connector is located on the MMC board as shown in Figure 5.

8/22

Figure 4: MMC position above the AMC board

7,3

8 m

m m

ax

20 mm

4 m

m

Component area

Max height Allowed on Single size AMCboard

Samtec SS4-20-3.00-L-D-L-K-TRhttp://www.samtec.com/ProductInformation/TechnicalSpecifications/CatalogPages.aspx?series=SS4

1 m

m

4.8

mm

AMC

Con

3.0

mm

Component area

1.2

mm

Samtec ST4-20-1.00-L-D-L-P-TRhttp://www.samtec.com/ProductInformation/TechnicalSpecifications/CatalogPages.aspx?series=SS4

MMC

Figure 5: Meaanine dimensions (top view)

MMC mezzanine board specifications, J.P.Cachemiche Ref.: Issue 3; Revision 0

MMC hardware implementation Date: 31 March 2011

9/22

MMC mezzanine board specifications, J.P.Cachemiche Ref.: Issue 3; Revision 0

MMC hardware implementation Date: 31 March 2011

3.2. Functionalities

The board will be based on a Atmega128L_8AU microcontroller from Atmel. Most of the interconnections with the 40 pins connector will be made with the help of its GPIO.

The global connectivity with the AMC board is shown in Figure 6.

The main functions of the MMC include:

• Monitoring of ejector switch for hot swap functionality• Providing IPMI interface over the IPMB_L channel• Monitoring of two FPGAs and board temperatures with critical and non-critical

alerting• Monitoring of the payload powers• Remote reset and shutdown of the board• Providing FRU inventory information (board and serial numbers, manufacturer,

etc...), power requirements record and point-to-point connectivity record for proper e-keying.

• Control of three LEDs (Blue, Red, Green) on the front panel to provide basic feedback to the user about Hot Swap state of the Module, failure and out of service status.

10/22

Figure 6: MMC connections on an AMC board

Microcontroller(ATmega128L)

GPIO

GPIO

TWI

ADCEnable#+3VManagementPower

GA[0..2]

PS#[0..1]

µTCA

bac

kpla

ne

IPMB-L

ResetVCC

DCDCconverters

& regulators

FPGA FPGA

LEDs

Handle

+12V Payload Power

Power OK

TemperatureSensor

I2C

MMC Mezzanine

ReprogramFPGAs

Init_Done SlaveInit_Done Master

Enable

AMC board

MMC mezzanine board specifications, J.P.Cachemiche Ref.: Issue 3; Revision 0

MMC hardware implementation Date: 31 March 2011

The Atmega128L features the alternate functions (TWI/UART/ADC...) on dedicated port pins. Some software interfaces (I2C) are implemented on the General Purpose I/O (GPIO) pins.

The Two-wire (TWI ) multi-master serial interface of the ATmega128L is used to implement IPMB_L (Intelligent Platform Management Bus) interface for IPMI messaging between Carrier and Module. The address of the Module on the IPMB_L is defined by the states of three Geographical address lines( GA[0..2] ).

The PS0# and PS1# connected to the GPIO pins on ATmega128L are used to detect presence of the Module in the microTCA crate and shutdown the Module in case of a not proper extraction.

The MMC uses one channel of 10-bit resolution ADC to monitor Payload Power (+12V). Two pins of the GPIO are connected respectively to DC-DC converters and regulators, which allows to switch on/off and monitor all payload powers on the Module.

The software implemented I2C (Inter-Integrated Circuit) on the GPIO pins provides interface to the AMC board temperature sensor LM95231 and a serial EEPROM ATC24C located on the MMC board.

Three pins are used to control Blue, Red and Green LEDs placed on the front panel.

One pin is connected to the Hot Swap microswitch to get activation or extraction request from the user.

Several pins are connected to the payload electronics to control and monitor of the payload (FPGA, DC-DC converters).

3.3. Interface with the board

The Table 1 below describes all the signals present on the 40 pins connector described in Figure 3 page 7.

11/22

MMC mezzanine board specifications, J.P.Cachemiche Ref.: Issue 3; Revision 0

MMC hardware implementation Date: 31 March 2011

12/22

Table 1: MMC connector pinout

Name Direction Description

1 12v Input 12V from the backplane -3 LOCAL_FPGA1_INIT_DONE Input PC45 LOCAL_FPGA2_INIT_DONE Input PC57 LOCAL_I2C_SCL Output Clock of the i2C bus to the AMC PD49 LOCAL_I2C_SDA Data of the i2C bus to the AMC PD511 LOCAL_MICROSWITCH_CLOSED_N Input Active low signal when extraction handle closed, else open PD2

13 LOCAL_LOW_VOLTAGE_POK Input PA0

15 AMC_GA<1> Input PB217 AMC_GA<0> Input PB119 AMC_PS1_N Input -21 AMC_PS0_N Input -23 RTM_PS Input PA225 RTM_12V_ENABLE Output PA327 RTM_3V3_ENABLE Output PA429 GND - -31 RTM_I2C_ENABLE Output PA533 DIGITAL INPUT/OUTPUT ( 3,3V) PE635 DIGITAL INPUT/OUTPUT ( 3,3V) or EXT INTERRUPT INT4 PE437 DIGITAL INPUT/OUTPUT ( 3,3V) or EXT INTERRUPT INT5 PE539 MANAGEMENT_POWER Input -2 LOCAL_RESET_FPGA_N Output PC2

4 LOCAL_RELOAD_FPGAS_N Output PC3

6 LOCAL_GREEN_LED_N Output PB68 LOCAL_BLUE_LED_N Output Active low blue led signal PB710 LOCAL_RED_LED_N Output Active low red led signal PB512 GND - -14 AMC_IPMB_SCL_L Input Clock of the i2C bus from the backplane PD016 AMC_IPMB_SDA_L Data of the i2C bus from the backplane PD118 LOCAL_REG_ENABLE Output Open drain signal to the enable of all regulators PC620 LOCAL_DCDC_ENABLE Output Open drain signal to the enable of all DC/DC converters PC722 AMC_GA<2> Input PB3

24 AMC_ENABLE_N Input PE7

26 PF128 PF230 PF332 LOCAL_TCK Output JTAG CLK PG034 LOCAL_TMS Output JTAG TMS PG136 LOCAL_TDO Input JTAG TDO PG238 LOCAL_TDI Output JTAG TDI PG340 DIGITAL INPUT/OUTPUT ( 3,3V) PE7

Pinnumber

Atmega 128Port

« Init_Done » signal from FPGA 1 (3.0V)« Init_Done » signal from FPGA 2 (3.0V)

Bidir

Power OK signal resulting from a « and » of the the power OK of all regulators and converters (3.3V)Geographic address bit 1 from the backplaneGeographic address bit 0 from the backplaneActive low signal coming from the backplaneActive low signal to be connected to PS1_N on the mezzanineRTM presenceRTM 12V enableRTM MP enableGroundRTM I2C buffer enable

User defined BidirUser defined BidirUser defined Bidir

3,3V management power« Reset FPGA » signal to the reset of all FPGAActive low open drain signal to FPGAs to request to reload them from their respective Flash EEPROMActive low green led signal

Ground

Bidir

Geographic address bit 2 from the backplaneActive low signal coming from the backplane to enable AMC 12V payload power

User defined Bidir DIGITAL INPUT/OUTPUT ( 3,3V) or ADC(Vmax 3,3V)User defined Bidir DIGITAL INPUT/OUTPUT ( 3,3V) or ADC(Vmax 3,3V)User defined Bidir DIGITAL INPUT/OUTPUT ( 3,3V) or ADC(Vmax 3,3V)

User defined Bidir

MMC mezzanine board specifications, J.P.Cachemiche Ref.: Issue 3; Revision 0

MMC software implementation Date: 31 March 2011

4. MMC software implementation

The microcontroller software is based on an open source code. There are 2 parts in this code: a core code and a user code.

The core code assumes the board insertion and exchange protocol with the MCMC located on the MCH if the system is a µTCA crate, or the MCMC located on the ATCA carrier if this is an ATCA system.

The user code deals with the specific features implemented on the AMC card. It must be slightly adapted to take these features into account.

4.1. Core code

4.1.1 Led managementAll control LEDs' statuses can be read by IPMI commands.

The RED LED (Out-Of-Service ) indicates the state of the payload powers generated by DC-DC converters:

• Off = ok.• On = failure.

The GREEN LED (USER LED) indicates Stratix IV GX FPGA configuration process status:

• Off = not configured.• On = done.

The BLUE LED (Hot Swap) indicates the module’s state as it deactivates in preparation for extraction or reactivates after insertion.

Insertion Sequence:

• Off = module handle open. Management power is not enabled.• Blue on = module is fully seated in carrier. Module’s management power is enabled.

User may initiate activation by pushing in the handle on the module’s front panel to close the hot swap switch.

• Blue long blink = module handle is closed; module is being activated.• Off = module handle is closed. Module is in normal operational state.

13/22

MMC mezzanine board specifications, J.P.Cachemiche Ref.: Issue 3; Revision 0

MMC software implementation Date: 31 March 2011

Extraction Sequence:

• Off = module is in normal operational state. User may initiate deactivation by pulling out the module handle to open the hot swap switch, sending a request via the MMC to the carrier for a hot swap extraction.

• Blue short blink = module handle open. Module is waiting to be deactivated. Not safe to extract module.

• Blue on = module is quiesced, module payload power is disabled. Safe to extract module.

The state diagram corresponding to a hot plug insertion or extraction of the board is shown in Figure 7.

4.1.2 Payload managementThe MMC provides control and monitor over the Module's payload.

Control

14/22

Figure 7: Insertion and extraction sequence state diagram

MMC mezzanine board specifications, J.P.Cachemiche Ref.: Issue 3; Revision 0

MMC software implementation Date: 31 March 2011

The base level control of the payload is performed by MMC using:

• a reload signal to FPGAs allowing to reload the firmware from the associated EPCS flash EEPROMs.

• and an Enable signal to DC-DC converters for switch on/off payload power.The MMC provides "FRU Control" IPMI command to reboot and reset FPGA remotely.

Status

The MMC determines the status of the two Stratix IV GX configuration process reading signal (INIT_DONE) of each FPGA.

The signal POWER_GOOD from DC-DC converters and regulators is used to monitor the state of all powers generated by converters. The MMC provides these informations to the user by means of the sensors and LEDs.

4.1.3 Sensors managementThe MMC monitors on-board sensors to determine the status of the board and to take appropriate actions in the event of a catastrophic failure. All sensors' informations are stored in MMC's SDRs (Sensor Data Records) in accordance with PICMG AMC.0 specification and available to AMC Carrier by means of IPMI. The sensor data record content is shown in Table 2.

The AMC board can incorporate one digital 2-wire serial temperature sensor ( LM95231 for example). This sensor is used to measure the local board temperature and to monitor the internal temperature of 2 FPGAs if they have a temperature probe.

4.1.4 FRU informationOn request of the MCMC located on the MCH board, specific information must be sent by the MMC over the IPMB-L bus.

15/22

Table 2: Sensor data record

N° Name Type Description0 FRU Hot Swap discrete State of the Hot Swap latch (1-opened, 0-closed)1 Temp FPGA master analog2 Temp FPGA slave analog3 Temp I/O analog Board local temperature4 POWER_OK discrete5 FPGA master discrete6 FPGA slave discrete7 12V analog +12V Payload power

Temperature of the Stratix IV GX FPGATemperature of the Stratix IV GX FPGA

State of all payload powers generated by DC-DC converters (1-ok)State of master Stratix IV GX FPGA configuration (1-done)State of slave Stratix IV GX FPGA configuration (1-done)

MMC mezzanine board specifications, J.P.Cachemiche Ref.: Issue 3; Revision 0

MMC software implementation Date: 31 March 2011

Table 3 describes the organization of the 1 KB of FRU information stored in the 4KB on-chip EEPROM. This information can be retrieved through use of the standard IPMI Read FRU command.

Where:

• header contains the information of the FRU storage format specification and the byte offset to the other sections of the FRU information..

• board contains information about the board on which the FRU information device is located as shown in Table 4.

• product contains information about the FRU itself, as shown in Table 5.• multirecord contains module current requirements and E-Keying information for

the AMC-Sormiou board as shown in Tables 6 and 7.

16/22

Table 3: FRU information

Area Size, in bytesHeader 8Internal 0Chassis 0Board (calculated)Product (calculated)

(calculated)Total 1024Multirecord

MMC mezzanine board specifications, J.P.Cachemiche Ref.: Issue 3; Revision 0

MMC software implementation Date: 31 March 2011

17/22

Table 4: Board Information Area

Field description Size, in Default valueFormat version 1 0x01Board area length 1 (calculated)Language code 1 0x19 (English)Manufacturer date/time 3 (based on manufacture date)

1 0xe0Board manufacturer 32 CPPMBoard product name type/length 1 0xc7Board product name 7Board serial number type/length 1 0xccBoard serial number 12 (based on serial number)Board part number type/length 1 0xc9Board part number 9 AMC-SFRU file ID type/length 1 0xC0No more fields 1 0xC1Padding (calculated) 0x00

1 (calculated)Total (calculated)

Board manufacturer type/length

AMC-Sormiou

Board area checksum

Table 5: Product information area

Field description Size, bytes Default valueFormat version 1 0x01Product area length 1 (calculated)Language code 1 0x19 (English)Product manufacturer type/length 1 0xe0Product manufacturer 32 CPPMProduct name type/length 1 0xc7Product name 7Board part/model number type/length 1 0xc9Board part/model 9 AMC-SProduct version type/length 1 0xc2Product version 2 1Product serial number type/length 1 0xccProduct serial number 12 (based on serial number)Asset tag type/length 1 0xc0FRU file ID type/length 1 0xc0End of fields 1 0xc1Padding 1 0x00

1 (calculated)Total (calculated)

AMC-Sugiton

Product area checksum

MMC mezzanine board specifications, J.P.Cachemiche Ref.: Issue 3; Revision 0

MMC software implementation Date: 31 March 2011

18/22

Table 6: Multirecord area: Module current requirement record

Field description Size, in bytes Default valueRecord type ID 1 0xc0 ( OEM )End of list/version 1 0x02Record length 1 0x06

1 (calculated)1 (calculated)

3PICMG record ID 1 0x16 (Power Record)Current draw( in 100mA) 1 150(15A)Record format version 1 0x00Total 11

Record checksumHeader checksumManufacturer ID,Least significant byte (LSB) first

0x5A 0x31 0x00( PICMG ID 12634 )

Table 7: Multirecord area: AMC point-to-point connectivity record

Field description Size, in bytes Default valueRecord type ID 1 0xc0 ( OEM )End of list/version 1 0x82Record length 1 0x10

1 (calculated)1 (calculated)

3PICMG record ID 1 0x19 (Point to Point Record)Record format version 1 0x00OEM GUID count 1 0x00Record type 1 0x80AMC channel descriptor count 1 0x01

3 0xa4, 0x98, 0xf3AMC link descriptors, LSB first 5 to be definedTotal 21

Record checksumHeader checksumManufacturer ID,Least significant byte (LSB) first

0x5A 0x31 0x00( PICMG ID 12634 )

AMC channel descriptors, LSBfirst ( channel ID = 0)

MMC mezzanine board specifications, J.P.Cachemiche Ref.: Issue 3; Revision 0

MMC software implementation Date: 31 March 2011

4.1.5 IPMI commandsThe MMC supports the commands listed in Table 8.

4.2. User code

User code handles specific temperature sensors, fan command and whatever specific function is implemented on the board. Templates are provided for few devices. Users have to adapt the code for their own use.

19/22

Table 8: IPMI commands

Message command SpecificationGet Device ID IPMI App (0x06) 0x01Broadcast “Get Device ID” IPMI App (0x06) 0x01Set Event Receiver IPMI Sensor/Event(0x04) 0x00Get Event Receiver IPMI Sensor/Event(0x04) 0x01Platform Event (Event Message) IPMI Sensor/Event(0x04) 0x02Get Device SDR Info IPMI Sensor/Event(0x04) 0x20Get Device SDR IPMI Sensor/Event(0x04) 0x21Reserve Device SDR Repository IPMI Sensor/Event(0x04) 0x22Get Sensor Reading IPMI Sensor/Event(0x04) 0x2DGet FRU Inventory Area Info IPMI Storage (0x0A) 0x10Read FRU Data IPMI Storage (0x0A) 0x11Write FRU Data IPMI Storage (0x0A) 0x12Get PICMG Properties PICMG Group extension(0x2C) 0x00FRU Control PICMG Group extension(0x2C) 0x04Get FRU LED Properties PICMG Group extension(0x2C) 0x05Get LED Color Capabilities PICMG Group extension(0x2C) 0x06Set FRU LED State PICMG Group extension(0x2C) 0x07Get FRU LED State PICMG Group extension(0x2C) 0x08Get Device Locator Record PICMG Group extension(0x2C) 0x0DSet AMC Port State AMC Group extension(0x2C) 0x19Get AMC Port State AMC Group extension(0x2C) 0x1A

Netfn Cmd

MMC mezzanine board specifications, J.P.Cachemiche Ref.: Issue 3; Revision 0

MMC software implementation Date: 31 March 2011

Bibliography1: PICMG, PICMG Advanced Mezzanine Card AMC.0 Specification R1.0 ECR-002 D0.8, April 28, 2006

2: Intel, IPMI Intelligent Platform Management Bus Communications Protocol Specification, v1.0Document Revision 1.0, November 15, 1999

3: Intel, Intelligent Platform Management Interface Specification Second Generation v2.0, February, 2006

4: Sept 27, 1999, IPMI Platform Management FRU Information Storage Definition v1.0 Document Revision 1.1,

20/22

MMC mezzanine board specifications, J.P.Cachemiche Ref.: Issue 3; Revision 0

MMC software implementation Date: 31 March 2011

List of FiguresFigure 1: Hardware Platform Management in a µTCA system .................................................5

Figure 2: Samtec FTE male connector ............................................................................................7

Figure 3: Implémentation approximative de la carte MCM .......................................................8

Figure 4: MMC connections with AMC board .............................................................................9

Figure 5: Insertion and extraction sequence state diagram ......................................................13

21/22

MMC mezzanine board specifications, J.P.Cachemiche Ref.: Issue 3; Revision 0

MMC software implementation Date: 31 March 2011

List of TablesTable 1: MMC connector pinout .....................................................................................................................................7

Table 2: FRU information ..............................................................................................................................................13

Table 3: Board Information Area ..................................................................................................................................14

Table 4: Product information area ................................................................................................................................14

Table 5: Multirecord area: Module current requirement record ..............................................................................15

Table 6: Multirecord area: AMC point-to-point connectivity record ......................................................................15

Table 7: IPMI commands ...............................................................................................................................................16

22/22