Embed Size (px)
Citation preview
TécnicasdeInteracçãoAvançadasBITalinoSensors
BITalino• h/p://bitalino.com/
• Bitalinoisasmallsensorpla8ormwellsuitedforworkingwithphysiologicaldata• CommunicaAonviaClassIIBluetoothv2.0• Availablesensors:
• EMG• ECG• Accelerometer• Light• EDA
BitalinobaseboardMicro-ControllerThe micro-controller unit (MCU)block converts the analogicsignals from the sensors to adigital format, and samples all ofthe channels. The MCU providesaccess to theBITalinoanaloganddigital channels,aswell as to theperipherals.
PowerThe power management blockprovides energy to all the otherBITalino blocks. Thismodule alsohasabuilt-inchargerthatcontrolsthe ba/ery charging. Charging ismade when the device is turnedoffbyconnecAngapower supplyto theMicro-USBport.Thisblockalso provides access to controlsignals
BluetoothA Class II Bluetooth v2.0 with arangeupto10m
Sensors:EMG
The sensor is capable of performingelectromyography (EMG) measurementsusing bipolar surface electrodes (plus aground lead), and monitors the muscleacAvaAon.
Sensors:ECG
The ECG works mostly by detecAngand amplifying the Any electricalchangeson the skin thatare causedduringtheheartmusclecycleduringeachheartbeat.
Sensors:ACC
T h i s s e n s o r c a n m e a s u r eacceleraAonsrelaAvetofreefallandthe model available is capable ofdetecAngmagnitudeanddirecAonofthis same acceleraAon, as a vectorquanAty. This resulAng vector canthen be used to sense posiAon,vibraAon,shock,fall,etc.
Sensors:EDA
Electrodermal AcAvity (EDA) can bedefined as a transient change incertain electrical properAes of theskin,associatedwiththesweatglandacAvity and elicited by any sAmulusthat evokes an arousal or orienAngresponse.
Sensors:LUX
A pho tod iode i s a t ype o fphotodetectorcapableofconverAnglightintoeithercurrentorvoltage.Inthis case the output is given involtage.
Actuators:LED
TheLEDmoduleconsists inasimpleLight-EmiUngDiode(LED)thatturnsonwhentheinputsignal ishighandturns off when the input signal islow.
BitalinoEnablingSoftware• OpenSignals(r)evoluAon
• SoWwarefordataacquisiAon,visualizaAonandprocessingwhileusingPLUX’sbiosignalpla8orms
• AllowstheusertoacquiredatafromoneormulApledevices,setuptheexactacquisiAonparameters
• Save standard ASCII text format or in the more efficient Hierarchical DataFormat(HDF)forprocessingbythirdpartytools
BitalinoSoftwaredevelopment• MulApleProgramminglanguages• Java(Desktopandandroid)• C++• C#• BonsaiLibrary• LabVIEW• MATLAB• Etc.
BitalinoJavaAPI(versionPriscilaAlves)• Pre-Requisites(JavaWindows)• JavaAPI• EclipseMars(recommended)• Bluecove-2.1.1-SNAPSHOT(bluetoothcommunicaAon)• PCwithbluetoothdevice
• PairBitalinodevicewithcomputer(pass:1234)
BitalinoAPI
BITalinoAPI
BITalino
BITalinoErrorTypesBITalinoExcepAon
DeviceDiscoverer
Frame
SensorDataConverter
BitalinoClass
YourApplicaAon
BitalinoObject
FramesConfig
BlueCovebluetoothPCbluetoothHardware
BitalinoClass• MainfuncAons:• Open–bitalino.open(StringmacAdd,intsamplingRate);• Start–bitalino.start(int[]anChannels);• Stop– bitalino.stop();• Close–bitalino.close();• Read-bitalino.read(intnSamples)• Write–bitalino.write(intdata);• Trigger–bitalino.trigger(int[]digitalArray);
DataStructure
DataacquiredwithNframes
ValuesfromacAvechannels
Analogchannelrawdata
MyFirstBitalino• BasicExample• Connect• StartandStopStream• CollectandShowdatafromdifferentsensors
• SimplisAcrecognizer(EMG+LED)
MyFirstBitalino1. NewAndroidApplicaAonProject2. ImportBitalinoTest3. ProperAes->JavaBuildPath->Libraries->AddExternalJars
I. bluecove-2.1.1-SNAPSHOT.jar;4. OrderandExporttab
MyFirstBitalino• Imports:
importBITalino.BITalino;importBITalino.Frame;importBITalino.SensorDataConverter;[email protected];importjavax.bluetooth.RemoteDevice;
MyFirstBitalino• Somevariablesneeded:publicstaAcFrame[]dataAcquired;//setofframes(nSamples)privatestaAcfinalintACC_MIN=208;privatestaAcfinalintACC_MAX=312;//Accelerometerconverter
MyFirstBitalinoSensorDataConvertersc=newSensorDataConverter();//classthatconvertsrawdataintomeaningfuldataint[]aChannels={0,1,2};//variablethatdefinesfromwhichchannelacquiredata
MyFirstBitalino• IniAalizesensorBITalinodevice=newBITalino();//finddevicesVector<RemoteDevice>devices=device.findDevices();System.out.println(devices);//connecttoBITalinodeviceStringmacAddress="98:D3:31:90:3D:C2";intsamplingRate=1000;Intsamples=2000;device.open(macAddress,samplingRate);
MyFirstBitalino• Acquiredata:
//startacquisiAononanalogchannels0to2device.start(aChannels);dataAcquired=device.read(samples);for(Frameframe:dataAcquired){
System.out.println("rawdataaccX:"+frame.analog[0]+"rawdataaccY:“+frame.analog[1]+"rawdataaccZ:"+frame.analog[2]);
System.out.println("converteddataaccX:"+sc.scaleAccelerometerWithPrecision(0,frame.analog[0],ACC_MIN,ACC_MAX)+"converteddataaccY:“+sc.scaleAccelerometerWithPrecision(1,frame.analog[1],ACC_MIN,ACC_MAX)+"converteddataaccZ:"+sc.scaleAccelerometerWithPrecision(2,frame.analog[2],ACC_MIN,ACC_MAX));}
MyFirstBitalino• Closedataacquisi.onandbluetoothconnec.on:
//stopacquisiAondevice.stop();//closebluetoothconnecAondevice.close();
Exercises
sEMGsensor• "Electromyography(EMG)isanexperimentaltechniqueconcernedwiththedevelopment,recordingandanalysisofmyoelectric signals.
• Myoelectric signals areformed byphysiological variaAons inthe state ofmusclefibermembranes."
sEMGsensor• Typicalbenefitsare:• EMGallowstodirectly“look”intothemuscle• Itallowsmeasurementofmuscularperformance• Helpsindecisionmakingbothbefore/aWersurgery• Documentstreatmentandtrainingregimes• HelpspaAentsto“find”andtraintheirmuscles• AllowsanalysistoimprovesportsacAviAes• Detectsmuscleresponseinergonomicstudies
sEMGsensor• HowmusclecontracAonworks?
sEMGsensor
sEMGsensor• sEMGsignal
sEMGsensor• FactorsinfluencingtheEMGsignal:• TissuecharacterisAcs• Physiologicalcrosstalk• Changesinthegemoetrybetweenmusclebellyandeletrodesite• Externalnoise• Electrodeandamplifiers
sEMGsensor• HumanAnatomy
Howtobuildarecognizer• Capturingthesignalofadatastream• WindowingandOverlapping
Howtobuildarecognizer• Pre-processingthesignal• Bu/erworthlow-pass,high-passandbandpassfilters
• Frequenciesdependonthesensorused• Helpsfilteringunwantedsignalsandenhancethesignalquality(e.g.Noisefromotherdevices,etc.)
Howtobuildarecognizer• FeatureExtracAon• Derivedvaluesfromthesignal• Transformlargesetsofdataintoareducedsetoffeatures• HelpsclassificaAonalgorithmstofindpa/ernsindata• ExamplesforEMGsensor:
• Waveformlength,RootMeanSquare,Wamp
Howtobuildarecognizer• Classifier• IdenAfyingtowhichofasetofcategories(sub-populaAons)anewobservaAonbelongs
• SupervisedLearning
WL RMS WAMP Gesture
100 200 300 WristFlexion
150 200 400 Closehand
400 100 100 Openhand
... ... ... ...
Howtobuildarecognizer
“Closehand”
WL RMS WAMP Gesture
100 200 300 WristFlexion
150 200 400 Closehand
400 100 100 Openhand
... ... ... ...
WL RMS WAMP Gesture
150 200 400 ?
Pre-processing
FeaturesExtrac@on ClassifierCapture
WEKAAPI• WekaisacollecAonofmachinelearningalgorithmsfordataminingtasks• SVM,RandomTree,BayesNet,etc.• h/p://www.cs.waikato.ac.nz/ml/weka/index.html
WEKAAPI• DataSctructure//DeclaretwonumericaWributesA]ributeA]ribute1=newA]ribute(“firstNumeric”);A]ributeA]ribute2=newA]ribute(“secondNumeric”);//DeclareanominalaWributealongwithitsvaluesFastVectorfvNominalVal=newFastVector(3);fvNominalVal.addElement(“blue”);fvNominalVal.addElement(“gray”);fvNominalVal.addElement(“black”);A]ributeA]ribute3=newA]ribute(“aNominal”, fvNominalVal);//DeclaretheclassaWributealongwithitsvaluesFastVectorfvClassVal=newFastVector(2);fvClassVal.addElement(“posi@ve”);fvClassVal.addElement(“nega@ve”);A]ributeClassA]ribute=newA]ribute(“theClass”,fvClassVal);//DeclarethefeaturevectorFastVectorfvWekaA/ributes=new FastVector(4);fvWekaA]ributes.addElement(A]ribute1);fvWekaA]ributes.addElement(A]ribute2);fvWekaA]ributes.addElement(A]ribute3);fvWekaA]ributes.addElement(ClassA]ribute);
WEKAAPI• Trainaclassifier//CreateanemptytrainingsetInstancesisTrainingSet=newInstances("Rel",fvWekaA]ributes,10);//SetclassindexisTrainingSet.setClassIndex(3);//CreatetheinstanceInstanceiExample=newDenseInstance(4);iExample.setValue((A]ribute)fvWekaA]ributes.elementAt(0),1.0);iExample.setValue((A]ribute)fvWekaA]ributes.elementAt(1),0.5);iExample.setValue((A]ribute)fvWekaA]ributes.elementAt(2),"gray");iExample.setValue((A]ribute)fvWekaA]ributes.elementAt(3),"posi@ve");//addtheinstanceisTrainingSet.add(iExample);//CreateanaïvebayesclassifierClassifiercModel=(Classifier)newNaiveBayes();cModel.buildClassifier(isTrainingSet);
WEKAAPI• TesttheClassifier//TestthemodelEvalua@oneTest=newEvalua@on(isTrainingSet);eTest.evaluateModel(cModel,isTes@ngSet);//PrinttheresultàlaWekaexplorer:StringstrSummary=eTest.toSummaryString();System.out.println(strSummary);//Gettheconfusionmatrixdouble[][]cmMatrix=eTest.confusionMatrix();
WEKAAPI• Usetheclassifier//setclassaWributeunlabeled.setClassIndex(unlabeled.numA]ributes()-1);//createcopyInstanceslabeled=newInstances(unlabeled);//labelinstancesfor(inti=0;i<unlabeled.numInstances();i++){doubleclsLabel=cModel.classifyInstance(unlabeled.instance(i));labeled.instance(i).setClassValue(clsLabel);}
References• DavidCostaandCarlosDuarte,"FromOnetoManyUsersandContexts:aClassifier
formarmandhandgestures",IUI'15,Atlanta,USA,2015• MakingMuscle-ComputerInterfacesMorePracAcal.T.Sco/Saponas,DesneyS.Tan,
DanMorris,JimTurner,andJamesA.Landay.Proceedingsofthe 2010SIGCHI(Atlanta,GA,April10-15,2010).CHI'10.ACM,NewYork,NY,851-854
• EnhancingInputOnandAbovetheInteracAveSurfacewithMuscleSensing.HrvojeBenko,T.Sco/Saponas,DanMorris,andDesneyS.Tan.ProceedingsofACMInteracAveTabletopsandSurfaces(Banff,Alberta,Canada,November23-25,2009).ITS'09.ACM,NewYork,NY,93-100
• EnablingAlways-AvailableInputwithMuscle-ComputerInterfaces.T.Sco/Saponas,DesneyS.Tan,DanMorris,RavinBalakrishnan,JimTurner,andJamesA.Landay.ProceedingsACMSymposiumonUserInterfaceSoWwareandTechnology(Victoria,BC,October4-7,2009).UIST'09.ACM,NewYork,NY,167-176.
• MakingGesturalInputfromArm-WornInerAalSensorsMorePracAcal.LouisKratz,T.Sco/Saponas,DanMorris.ProceedingsofSIGCHIConferenceonHumanFactorsinCompuAngSystems(May2012).CHI'12.ACM,NewYork,NY,1747-1750.
• RecoFit:usingawearablesensortofind,recognize,andcountrepeAAveexercises.DanMorris,T.Sco/Saponas,AndrewGuillory,andIlyaKelner.ProceedingsoftheSIGCHIConferenceonHumanFactorsinCompuAngSystems(April2014).CHI'14.ACM,NewYork,NY,3225-3234.
