Artificial Intelligence Research at Keen Software House

Technical Report

Shortly about AGI

• Artificial General Intelligence

- Autonomous agent

- Able to perceive and change its environment

- Able to remember, reason and plan

- Adaptable and able to learn

- Able to communicate

What to use for AGI?

• Classical AI?- Symbolic architectures- Inference machines, expert systems- Planners and solvers, STRIPS

• Artificial neural networks?- Spiking Networks- FFN, RNN- DeepNets

• Multi agent systems?• All of them!

Suitable tool for experiments

• Rapid model prototyping - Integrate existing model- Create (or recreate) new model

• Model insight- Rich GUI & Visualization possibilities- Model structure view (oriented graph?)- Runtime view & execution control

• Heterogeneous architecture- Connect different models together- Able to use various hardware

• Parallel execution- GPU based solution- Cluster solution

Existing tools & inspiration

• Graph of connected modules- ROS

- Matlab / Simulink

- Maya material editor

- Nengo (Eliasmith)

• Specialized libraries (modules)- Caffe, OpenCV, cuBLAS, cuDNN,

- ROS modules

Our solution – Brain Simulator

• Model structure- Nodes, tasks, memory blocks, worlds

• Model view- Graph view (model structure)- Observers (model data)

• simple, numeric, 3D, custom

• Experiments & debugging - Model parameters exposed to GUI- Adjustable observers- Simulation control

• Parallel computing- CUDA (Intel Phi support in progress)- Multi GPU support

Brain Simulator – screenshots

Brain Simulator – modules

• Implemented modules- Feed-forward nets (FFN, RNN, convolution nets, auto associators)- Self-organizing networks (SOM, GNG, K-means …)- Vector symbolic architectures (HRR, BSC)- Hierarchical temporal memory (spatial & temporal poolers)- Spiking networks & STDP- Computer vision (filters, segmentation, tracking, optical flow)- Hopfield network, SVD, SLAM, PID, Differential evolution and many

others

• Imported modules- Caffe, BLAS, BEPU Physics, Space Engineers, Gameboy emulator

• Planned modules- Deep learning & RBMs, Hierarchical Q-Learning

BS Screenshots – SOM

Development methodology

• Iterative/agile approach- Early implementation and experiments- Separated experiments with mockup parts- Milestone oriented (global model iterations)

• Separated experiments (proofs of concept)- Data representation, memory models, temporal data encoding- Learning strategies, goal inference, action selection- Spatial awareness, visual working memory, navigation- Computer vision

• Milestone examples- 6-legged robot agent (integration test)- Breakout/Pong game (reinforcement learning & vision test)- Autonomous agent game (PacMan, Nethack)

Example 1 – walking robot

• Physical world emulation - Connected to Space Engineers game

- 6-legged robot body

- Runtime visual data processing & body control

• Learning from mentor - Hardwired movements

- Learning body state associated with high level movement commands

- Simple vision to action associations

- Totally supervised system

Video of 6-legged robot

Example 2 – Pong / Breakout

• Pong / Breakout game- From bitmap to buttons- Reinforced learning (reward and punishment)- Image processing towards object tracking- Vector symbolic architecture - Goal states extraction - Action learning & action selection

• Existing solutions- Not Q-learning (DeepMind and others before them)- Modular, engineered system- Better insight (faster learning?), sacrificed flexibility

Pong / Breakout model

Visual Processing

Pong / Breakout model

Pong / Breakout BS inspection

Future work

• Next milestone – 2D egocentric game- Advanced visual working memory- Navigation & inner spatial representation of environment- Environment variables extraction, hierarchical Q-learning- Multiple goals and motivations, goal chaining- Motoric systems (bipedal balancing)

• Future milestones- Same model playing different games - Same model instance playing different games- Motoric systems (command sequences unrolling & execution)

• Computing platform improvements- Brain Simulator release (with remote module repository)- HPC solution- Unix systems release

The end

• You can invest in AI companies• Every $1 invested today will return 1,000,000 times

• Join our team – we are always hiring• AI Programmers / Researchers• SW Engineers / Architects• PR Manager / Evangelist

• Follow us: • http://blog.marekrosa.org/• http:// www.keenswh.com/

Thank you.Questions?