Embed Size (px)
Citation preview
LVE Stats 2 Training
• Modular, extendable and efficient
architecture
o Can monitor anything
o Customize what is monitored
Use RES mem instead of LVE PMEM
o In memory data analysis - better
performance
o Easy to extend - suspend, notify, etc…
Why LVE Stats v2?
• Better precision
o Only 1 month of data vs 2 months for v1
o Yet, 1 minute intervals vs 1 hour intervals
Why LVE Stats v2?
• Better charts (SVG)
• Include DB Governor statistics
Why LVE Stats v2?
• Faults for CPU/IOo Not ‘faultable’ resources, just throttled
o We count it as a ‘fault’, once per minute, if limit reached
o Easier to identify/see when user hits the limit
Why LVE Stats v2?
• Collects snapshots
o URL
o Process
o Query
Why LVE Stats v2?
• Backwards compatible by default
o By default lve-stats-2.x runs in backward
compatibility mode.
lveinfo and lvechart run as before
Will be changed in ~12 months
• Make new mode default by setting mode=v2 in
/etc/sysconfig/lvestats2
o Or use -m v1|v2 option in lveinfo.
Backwards compatible
• lveinfo/lvechart are backward
compatible
o Old options should work the same
• /var/lve/info populated as before
o Every 5 seconds, used by control panel to
display ‘live’ information
What stayed the same?
• --dbgov -- allows to manipulate DB Governor stats
• -c/--csv [FILE] -- saves data in CSV format
• -j/--json -- json output
• --time-unit -- averages time in output based on the
time unit
• --servers-info for multi-server install, lists servers &
LVE version
• -m v1/v2 -- v1 display output in backward
compatible mode, v2 - new mode
• --blank-value [val] -- substitute instead of
unsupported limit, default -
lveinfo
• -u/--user -- show stats for particular MySQL user
• -l/--limit -- max number of entries in output
• -o/--order-by -- order output by
con/cpu/read/write
• -b/--format -- columns to show
• --csv/--json -- display data in csv or json formats
• --from/--to/--period -- same as lveinfo
lveinfo --dbgov
• Includes DB Governor data on CPU/IO charts
• --format {svg,png} -- allows to use SVG or PNG format, default SVG
What’s new: lvechart
• Top like utility for LVEo will replace lvetop
• Powers new UI
• Includes DB Governor data
• Supports --json only for now (more later, including live/on screen)o Used in the next version UI for cPanel/DA/Plesk
• Takes data from:o /var/lve/cloudlinux_top.json file
o Updated by lvestast v2 every 5 seconds
What’s new: cloudlinux-top
• - hide-mysql - don’t show DB Governor related data
• - u/--username - show data only for specific user,
substring matching
• - d/--domain - show data only for specific domain,
substring matching
• - m/--max - max number of entries to show, default 25
cloudlinux-top options:
• dbgovchart - show DBGovernor charts
• lve-create-db -- creates or re-creates database
for LVE Stats v2.0
• lve-read-snapshot -- reads snapshots for end
user. Can produce data in json, or produce
‘summary’ stats for a particular period
More commands...
• Not on CL5
Storing History: Supported Databases
• Automated upgrade from lve-stats 1.x
once released to production channel
• Automated settings migrations
• Data migrated automatically within first
24 hourso Throttled, so disks are not overloaded
o No support for migrating stats on centralized
storage
Upgrading: Migrating data
• /var/lve/lvestats2.db -- SQLite db
with all the data
• /var/lve/info -- ‘real time’ usage to
show in control panels
• /var/lve/cloudlinux_top.json -- data
for cloudlinux-top command
Important files
• /etc/sysconfig/lvestats2 -- main config file
• /etc/sysconfig/lvestats.config/ -- config
files for Pluginso HistoryCleaner.cfg -- how often to remove
data from db
o SnapshotSaver.cfg -- snapshot settings (how
many, how often)
o StatsNotifier.cfg -- Notifications settings
Config files
• db_type = sqlite
• plugins=/usr/share/lve-stats/plugins
• keep_history_days=30
• server_id=7269a505-3
• mode=v1
...
/etc/sysconfig/lvestats2
• NOTIFY_ADMIN=N
• NOTIFY_RESELLER=N
• NOTIFY_CUSTOMER=N
• NOTIFY_INCLUDE_RESELLER_CUSTOMER=N
• NOTIFY_CPU=Y
• NOTIFY_IO=Y
• NOTIFY_IOPS=Y
• NOTIFY_MEMORY=Y
• NOTIFY_EP=Y
• NOTIFY_NPROC=Y
• NOTIFY_MIN_FAULTS_ADMIN=1
• NOTIFY_MIN_FAULTS_USER=1
• NOTIFY_INTERVAL_ADMIN=12h
• NOTIFY_INTERVAL_USER=12h
StatsNotifer.cfg – Notify when limited
Notification message example
• period_between_incidents=300
• snapshots_per_minute=2
• max_snapshots_per_incident=10
SnapshotSaver.cfg
• Easy way to extend lvestats
• Simple Architecture
• Python based
• http://docs.cloudlinux.com/index.html?creating_a
_plugin.html
Plugins
• Collectors
• Analyzers
• Persistors
• Notifiers
collectors
analyzers
PersistorsNotifiers
Plugin Types
• Generic / always included
• Optional
o LVEDestroyer
o ResMEMCollector
• Custom
More about plugins
• period -- how often to execute, default 5
seconds
• order -- order in the execution sequence,
to decide which plugin goes next
• Any other settings can be set using config
files
Plugin Properties
Name Period Order Default
LVECollector 5 1000 Y
CPUInfoCollector 5 2000 Y
LVEUsernamesCollector 3600 3000 Y
LVEUsageAnalyzer 5 4000 Y
LveUsageAggregator 60 5000 Y
DBGovSaver 5 6000 Y
FileSaver 5 7000 Y
CloudLinuxTopFileSaver 60 8000 Y
Plugins
DBSaver 60 9000 Y
DbUsernamesSaver 3600 10000 Y
DBSaverX60 3600 11000 Y
SnapshotSaver 30 12000 Y
StatsNotifier varied 13000 Y
HistoryCleaner 3600 14000 Y
ResMEMCollector 30 1500 N
LVEDestroyer 5 - N
Plugins
Name Period Order Default
• To enable plugin, copy or link it to /usr/share/lve-
stats/plugins director
• For example to enable ResMEMCollector plugin, do
ln -s
/opt/alt/python27/lib/python2.7/site-packages/lvestat
s/plugins/generic/res_mem_collector.py
/usr/share/lve-stats/plugins/
Adding plugins
# FSize_watcher_collector.py# Example plugin for monitoring file size.# Part 1. Collector import osfrom lvestats.core.plugin import LveStatsPlugin # Key nameCOLLECTOR_KEY = 'FSizeWatcher_fsize'COLLECTOR_KEY_FILENAME = 'FSizeWatcher_fname' class FSize_watcher_collector (LveStatsPlugin):
# this plugin should be first in chain order = 0 # File to monitoring file_to_monitoring = None def __init__(self): pass # Sets configuration to plugin def set_config(self, config): self.file_to_monitoring = config.get('file_to_monitoring', None) pass # Work method def execute(self, lve_data): try: # if monitoring file absent, do nothing if self.file_to_monitoring is None or not os.path.exists(self.file_to_monitoring): return # Get file size stat_info = os.stat(self.file_to_monitoring) fsize = stat_info.st_size # Place file name and file size to server data dictionary lve_data[COLLECTOR_KEY_FILENAME] = self.file_to_monitoring lve_data[COLLECTOR_KEY] = fsize except (OSError, IOError): # file absent or any other error - remove size from dictionary del lve_data[COLLECTOR_KEY]
Example Plugin
• LVE_VERSION LVE version. • stats -- Dictionary, that contains lve id’s as keys and LVEStat class objects as values. Every LVEStat
object contains values of usages and limits taken from /proc/lve/list file for every LVE Id. Dictionary keys – integer lve id, including 0 for “default” LVE. This dictionary is updated on each iteration of lvestats-server (every 5 seconds by default).LVEStat – is a standard server class, it can be imported with the command from lvestats.core.lvestat import LVEStat.The class is described in the file /opt/alt/python27/lib/python2.7/site-packages/lvestats/core/lvestat.py.Here you can find the whole list of data fields and their functions.
• old_stats -- stats content from the previous iteration. Before the first iteration – empty dictionary.• totalHz -- When LVE_VERSION is 4, real CPU frequency in Hz multiplied by number of cores. When
LVE_VERSION > 4, CPU speed is in conventional units and equals to 1000000000 * cores (1 GHz per core).
• procs -- Quantity of CPU/cores.• lve_usages -- Contains accumulated LVE statistics for each 5-seconds interval in current minute. Cleared
each minute.• lve_usage -- Contains aggregated LVE Statistics for “previous” minute to store to database. Overwritten
each minute.
Plugin Data Convention
• Python 2.7 (www.python.org): No comment;
• Sqlite (sqlite.org), MySQL (www.mysql.com), MariaDB (
mariadb.org), PostgreSQL (www.postgresql.org)
• Sqlalchemy (sqlalchemy.org): the Python SQL Toolkit and
Object Relational Mapper;
• Alembic (alembic.zzzcomputing.com): database migration tool
for usage with the SQLAlchemy;
• Numpy (numpy.org): package for scientific computing with
Python;
• Sentry (sentry.io): real-time error tracking;
• Prospector (prospector.landscape.io): python static analysis
tool.
Stack technology
Questions?
Twitter.com/CloudLinuxOS Visit CloudLinux.com
