1. 2015 VMware Inc. All rights reserved. VMware App Volumes
Troubleshooting Denis Gundarev Sr. MTS/Application Architect, App
Volumes R&D
2. About me C:>whoami /all USER INFORMATION ----------------
User Name Twitter E-Mail ================ ============
================== VMWAREdgundarev @fdwl [email protected] GROUP
INFORMATION ----------------- Group Name Type SID
======================================== ================
================= BUILTINGeeks Mandatory group S-1-5-32-540
Mandatory LabelCrazy Russian Label S-1-16-8192 VMWAREApp Volumes
R&D Mandatory group S-1-5-32-544
3. App Volumes Dynamic Delivery Benefits Dynamic delivery of
managed application containers in seconds. Real time native
application and data delivery. Agility Logically manage application
sets based on business needs. Deliver or upgrade application sets
across all VMs in seconds. Simplicity Integrate into existing
infrastructure in minutes. Provision applications as easily as
installing them. Flexibility Persistent user experience with
non-persistent economics. Works with VMware Horizon 6 with View and
Citrix XenApp 6.5. Efficiency Optimize use of storage, SAN IOPS,
and network.
4. App Volumes Feb 3, 2015 - VMware App Volumes v2.5 Horizon
View integration Multiple templates for writable volumes Mar 12,
2015 - VMware App Volumes v2.6 AppStack Grouping Storage Groups Apr
28, 2015 - VMware App Volumes 2.7 One-way AD trusts Dynamic VHD
permissions XenDesktop 7.x support Jun 15, 2015 - VMware App
Volumes 2.9 Support for application deployment to physical machines
Multi vCenter Configurations
5. App Volumes components App Volumes Manager App Volumes Agent
AppStack volume(s) Console for assignments and configuration Broker
for App Volume Agent for the assignment of applications and
writable volumes File system and registry abstraction layer running
on the target system Virtualizes file system writes as appropriate
(when used with optional writable volume) Read-only volume
containing applications Can map more than one AppStack per user,
target Deploy apps to VDI or RDSH Writable volume Per user
read-write volume used to persist changes written in the session
One writable volume per user
7. App Volumes Operation Mode VMDK Direct Attached Operation
Mode preferred operations mode for App Volumes AppStacks and
Writable Volumes are stored within a hypervisor datastores VMDK
file format attached to the virtual machine using standard
hypervisor functionality. VHD In-Guest Operation Mode AppStacks and
Writable Volumes are stored on the (CIFS) file share VHD file
format attached to the target computer using OS functionality Works
with physical and other hypervisors
8. VHD In-Guest VHD AppStacks mounted inside the target
computer, similar to disk manager command Target computer should
have access to the file share VHD file permissions can be
dynamically controlled by the manager Works with physical and
third-party hypervisors
9. Hypervisor Connection Type Connection to VMware vSphere
vCenter preferred connection type for most environments VMDK Direct
Attached Operation Mode ESX (Single Host) Connection VMDK Direct
Attached Operation Mode works only for a single hypervisor host use
for small deployments and PoC VHD In-Guest disables hypervisor
connection enables use of VHD In-Guest Operation Mode CONFIDENTIAL
9
10. App Volumes - Scalability Observe vSphere 5.5 Maximums 2048
virtual disks per host 2048 powered-on virtual machines per VMFS
volume 60 SCSI devices per virtual machine Recommended Practice: 1
AppStack per 2,000 virtual machines Up to 20 AppStack volumes per
VM 10,000 Agents per App Volume Manager Core applications Part of
the base image or single AppStack AppStack datastore for every
2,000 users
11. AppStack App Volumes is not ThinApp Alternative ghost for
applications Works with ThinApp or App-V The contents of the system
drive (C:) located under SVROOT in the VHD. Only file system
entries that had contents changed / updated or security information
changed / updated. The files are ordered in the same directory tree
as in the source machine All the file system entries have the
original security information / attributes. The registry changes
are located in snapvol.dat and have a valid Windows registry hive
formatting. The changes / updates to the SOFTWARE hive are under
MACHINESOFTWARE key in the hive. The changes / updates to the
SYSTEM hive are under MACHINESYSTEM key in the hive. The changes /
updates to the DEFAULT hive are not captured as part of the
VHD.
12. Configuration rules
13. Workflows twitrcovers.com When Stuff Happens
14. App Volumes Provisioning Volumes Workflow Target System App
Volumes Manager vCenter 1. Configuration of AppStacks 2. Mount
Command sent to vCenter - Target system - VMDK information -
DataStore Information 3. Target system receive AppStacks 1 2 3
17. App Volumes Agent Shutdown App Volumes Manager Shutdown 1.
App Volumes Agents checks into App Volumes Manager 2. User based
assignments are detached 2 App Volumes Agent 1
18. App Volumes Agent Login App Volumes Manager Login 1. App
Volumes Agent checks into App Volumes Manager 2. App Volumes Agent
checks for pending attachments/un-attachments 3. App Volumes Agent
checks for machine based attachments if present then no user based
attachments are honored 4. App Volumes Manager checks database for
logged in user assignments 5. Attach assigned volumes - Writable
then AppStack(s) 2 App Volumes Agent 1 Database 3 4 5
19. App Volumes Agent Logout App Volumes Manager Logout 1. App
Volumes Agents checks into App Volumes Manager 2. User based
assignments are detached 2 App Volumes Agent 1
20. App Volumes Provisioning Start with Clean VM image Create
AppStack in App Volumes Manager Choose Provisioning VM VMDK gets
attached to Provisioning VM Install application Complete AppStack
creation VM restarts, VMDK is detached Ready to assign
AppStack
21. Scripts Script name Execution condition prestartup.bat
Called when a volume is dynamically attached or on during system
startup but before virtualization is activated startup.bat Called
when a volume is dynamically attached or on during system startup
startup_postsvc.bat Called as and called after services have been
started on the volume (not called if there are no services on
volume) logon.bat Called at logon and before Windows Explorer
starts logon_postsvc.bat Called after services have been started
(not called if there are no services on volume) shellstart.bat
Called when a volume is dynamically attached or when Windows
Explorer starts. shellstop.bat Called when the user is logging off
before Windows Explorer is terminated. logoff.bat Called at logoff
and Windows Explorer has terminated shutdown_presvc.bat Called when
the computer is being shutdown before services have been stopped.
shutdown.bat Called when the computer is being shutdown after
services have been stopped. allvolattached.bat Called after all
volumes have been processed (so if user has 3 AppStack, this will
be called after all 3 have loaded) post_prov.bat Called at the end
of provisioning to do any one-time steps that should be performed
at the end of provisioning. Invoked at the point of clicking the
provisioning complete pop-up while the volume is still virtualized.
prov_p2.bat Invoked at phase 2 of the provisioning process. After
the machine has rebooted but before App Volumes Manager has been
notified that provisioning has completed. This is the last chance
to perform any actions on the provisioned volume with
virtualization disabled.
22. Troubleshooting
23. App Volumes Agent Log files App Volumes Agent
C:Windowssvservice.log SvService responsible for communication with
App Volumes Manager, preparing volume, running post-attach scripts,
refreshing variables, registering fonts App Volumes Agent
C:WindowsSystem32LogFilesWMI AppVolumesAgent.etl Minifilter drivers
for NTFS Policy driven; the files, directories, registry keys and
processes that are virtualized are governed by the policy file
snapvol.cfg on each volume. App Volumes Agent svcapture.log
C:Windowssvcapture.log Perform the provisioning/editing functions,
generate policy files, metadata and report it to
Agent/Manager.
24. Reading SVDriver log Converting an ETL file to xml format
Flush buffers to disk logman update AppVolumesAgent -fd ets Convert
log to xml tracerpt.exe AppVolumesAgent.etl -of xml -gmt -tp
C:Program Files (x86)CloudVolumesAgenttmf -o AppVolumesAgent.xml
Open AppVolumesAgent.xml using Excel Event Viewer: Copy TMF files
from C:Program Files (x86)CloudVolumesAgenttmf to
C:windowsSystem32WinevtTraceFormat Open
"C:windowsSystem32LogFilesWMIAppVolumesAgent.etl using windows
Event Viewer
25. App Volumes Manager Production Log The Production.log file
can help identify issues You can view the Production.log at
http://:8080/log The default log level is set to info. Logfile is
also located at: C:Program Files (x86)CloudVolumesManagerLog
production.log
26. ESXi Logs ESXi Host Management Service Log (hostd)
/var/log/hostd.log Watch for disk locking, VM state transition,
hot-add operations Virtual Machine Log
/vmfs/volumes///vmware.log
27. Debug provides SQL logging App Volumes Manager Increasing
Logging Level We can increase the logging level to debug, by
editing: C:Program
Files(x86)CloudVolumesManagerconfiglog4r.yml
28. Example 1: Provision a new App Stack CONFIDENTIAL 30
Template VMDK copied to new App Stack VMDK UUID of clean machine to
mount App Stack for provisioning App Volumes module initializes
capture, RvSphere module mounts volume Operation: New App Stack
Office2010_RTM created Where to Look: App Volumes Manager Log
C:Program Files (x86)CloudVolumesManagerlogproduction.log
29. Example 1: Provision a new App Stack (continued)
CONFIDENTIAL 31 VM State transitions during operation: VM_STATE_ON
VM_STATE_RECONFIGURED VM_STATE_RECONFIGURING VM_STATE_ON. Hot Add
completed and App Stack mounted on VM Operation: New App Stack
Office2010_RTM created Where to Look: ESXi hostd.Log:
/var.log.hostd.log Relevant Log entries: Hot-Add, VM transition,
disk locking.
30. CONFIDENTIAL 32 Example 2: SQL Database Unreachable
Behaviour: Manager: UI is unavailable CVManager.exe service stays
up, attempting reconnection. ODBC error in Manager.log. Agent:
Where user remains logged in - App Stacks / Writables work as
normal For new logins - App Stacks / Writables work as normal
Issue: SQL Server Database is unreachable by App Volumes Manager(s)
Extract from App Volumes Manager.log Where to Look: App Volumes
Manager Log C:Program Files
(x86)CloudVolumesManagerlogproduction.log Relevant Module entries:
ODBC
31. CONFIDENTIAL 33 Example 3: AD Unreachable Issue: Active
Directory Domain Services unreachable (including DNS) Extract from
App Volumes Manager.log Behaviour: Manager: UI is functional, but
AD queries fail error in Manager.log Agent: Where user remains
logged in - App Stacks / Writables work as normal For new logins -
App Stacks / Writables unavailable Where to Look: App Volumes
Manager Log C:Program Files
(x86)CloudVolumesManagerlogproduction.log Relevant Module entries:
RADIR
32. CONFIDENTIAL 34 Example 4: vCenter Unreachable Issue:
vCenter unreachable (Note: Mount on ESXi host option was set during
install) Extract from App Volumes Manager.log Behaviour: Manager:
UI is partially functional, but error on querying datastores.
Unable to create new App Stacks/Writables Agent: App Stacks /
Writables fully functional Where to Look: App Volumes Manager Log
C:Program Files (x86)CloudVolumesManagerlogproduction.log Relevant
Module entries: RvSphere
33. App Volumes Manager Ruby Console The Ruby on Rails Console
can be used to troubleshoot, execute queries and run scripts Common
use to examine internal objects not visible in the UI Open command
prompt as Administrator on App Volumes Manager machine Navigate to:
C:Program Files (x86)CloudVolumesManager Enter the command:
rubybinruby.exe scriptrails console 35 Any ruby command should be
used with CAUTION ! ! !
34. App Volumes Manager Ruby Console List registered App Volume
Machines: ap Machine.all Show configuration ap
Svconfiguration.current Delete pending activities
Delayed::Job.destroy_all Search for machines ap Machine.where('name
LIKE ?', "PVS%") Delete all disabled computers ap
Computer.where("disabled_at IS NOT NULL").delete_all 36
35. App Volumes Manager Ruby Console List admin groups ap
GroupPermission.all Delete all administrators
GroupPermission.delete_all Add administrator group
Permission.named(:admin).add_group_name("CN=Domain
Admins,OU=Groups, DC=test,DC=com") 37
36. App Volumes Manager PowerShell Initiate session
$url="http://appvol01.corp.itbubble.ru" $person =
@{'user[account_name]'=admin'; 'user[password]'='P@ssw0rd'}
Invoke-RestMethod -Uri $url/login -Method POST -SessionVariable
session -Body $person Get version Invoke-RestMethod -Uri $url/login
-Method POST -SessionVariable session -Body $person Get online
users (Invoke-RestMethod -Uri $url/cv_api/online_entities
-WebSession $session -Method GET).online.records Get configuration
(Invoke-RestMethod -Uri $url/cv_api/summary -WebSession $session
-Method GET).current_configuration 38
38. AppVolumes PowerShell Module Preview Open-AppVolSession
-Uri http://manager.domain.com -Username admin -Password password1
Or you can omit the parameter names: Open-AppVolSession
http://manager.domain.com admin password1
39. AppVolumes PowerShell Module Preview Examples:
Get-AppVolAppStack [-All] - Returns all available appstacks
Get-AppVolAppStack -AppStackIds 88,19 return appstacks with IDs 88
and 19 88,19|Get-AppVolAppStack return appstacks with IDs 88 and 19
thru the pipe Get-AppVolAppStack -Name office returns all appstacks
where the name contains office Get-AppVolAppStack -Name office -Not
- returns all appstacks where the name NOT contains office
Get-AppVolAppStack -Name office -Exact all apstacks where the name
is exactly office Get-AppVolAppStack -Path "cloudvolumes" appstacks
that has cloudvolumes in the datastore path Get-AppVolAppStack
-DataStore iSCSI -Exact appstacks located on datastore iSCSI
Get-AppVolAppStack -FileName office appstacks where vmdk name
contains word office Get-AppVolAppStack -CreatedAt "4/28/2015" ge
appstacks created after or on 4/28/2015 Get-AppVolAppStack
-CreatedAt "4/28/2015" -gt appstacks created after but not on
4/28/2015 Get-AppVolAppStack -MountedAt $((get-date).AddDays(-30))
-ge -Not - appstacks not mounted in past 30 days Get-AppVolAppStack
-TemplateVersion "2.5.1" appstacks with template version 2.5.1
Get-AppVolAppStack -AssignmentsTotal 2 -ge - appstacks that have 2
or more assignments
40. AppVolumes PowerShell Module Preview Get-AppVolAppStackFile
-DataStore datastore1|Get-AppVolAppStack|Format- Table returns a
table with all appstacks that have files on datastore1
Get-AppVolAppStackFile -Reachable -Not -returns all unreachable
files Get-AppVolAppStack -Name office |Get-AppVolAssignment - get
all assignments for appstacks that has office in the name
Get-AppVolAssignment -EntityDn "cn=users,dc=domain,dc=com"
assignments for users in specific OU Get-AppVolAssignment
-EntitySamAccountName denis -Exact assignments for user denis
41. App Volumes Agent - Registry Entries
HKLMSoftwareCloudVolumesAgent
HKLMSystemCurrentControlSetservicessvserviceparameters vCenter /
App Volumes Manager Hostname and Port Additional App Volumes
Managers registry entries need to be added manually after standard
install Can be set during Msiexec install
MANAGER_ADDR=test.company.c om MANAGER_PORT=80
42. App Volumes & MSOffice
43. Microsoft Office Support Basics We dont officially support
scenarios that are not supported by Microsoft Office 2007 Mix of
64-bit and 32-bit apps We only support: Office Professional Plus
Office Standard Office 365 ProPlus (installed from Office
Professional Plus media) Only media downloaded from Microsoft
Volume Licensing Service Center (VLSC) can be used No MSDN No
Retail/OEM Verify if your installation media is correct: You must
have a folder named Admin on your ISO Name of the folder with .WW
extension (ProPlus.WW, Visio.WW, etc.) shouldnt have a lowercase r
before the dot - ProPlusr.WW, Visior.WW
44. Microsoft Office Activation KMS should be used for most
deployments MAK activation can be used for persistent deployments
starting with 2.7 Retail, OEM, Office 365 activation not supported
Active Directory-Based Activation (Office 2013 on Windows 8) may
work, but not tested KMS DNS discovery preferred Rearm Office after
installation If activation fails, verify that KMS activation works
for a machine with no App Volumes agent
45. Microsoft Office Recommendations Recommend Office in the
base image One AppStack for app Office Applications Never launch
Office Apps during provisioning Rearm Office after installation
Add-ins/plug-ins must be on the same AppStack with Office Recommend
32-bit editions 32-bit and 64-bit editions should never meet, even
from different AppStacks
46. Multiple Office Versions Within AppStack Office 2010 Office
2013 Project 2010 Project 2013 Visio 2010 Visio 2013 Office 2010
N/A Install Office 2010 first Supported Install Office 2010 first
Supported Install Office 2010 first Office 2013 Install Office 2010
first N/A Install Project 2010 first Supported Install Visio 2010
first Supported Project 2010 Supported Install Project 2010 first
N/A Install Project 2010 first Supported Install Project 2010 first
Project 2013 Install Office 2010 first Supported Install Project
2010 first N/A Install Visio 2010 first Supported Visio 2010
Supported Install Visio 2010 first Supported Install Visio 2010
first N/A Install Visio 2010 first Visio 2013 Install Office 2010
first Supported Install Project 2010 first Supported Install Visio
2010 first N/A
47. Native Office Coexistence AppStack Office 2010 Office 2013
Project 2010 Project 2013 Visio 2010 Visio 2013 Nativelyinstalled
Offic e 2010 Not Supported Supported except Outlook Supported
Supported Supported Supported Offic e 2013 Not Supported Not
Supported Not Supported Supported Not Supported Supported
