Commit Graph

147 Commits

Author SHA1 Message Date
Abhishek Kumar
2020bfb6a3 server: allows compute offering with or without constraints (#3245)
Problem: Custom compute offering does not allow setting min and max values for CPU and VRAM for custom VMs.

Root Cause: Custom compute offerings cannot be created with a given range of CPU number and memory instead it allows only fixed values.

Solution: createServiceOffering API has been modified to allow setting a defined range for CPU number and memory. Also, UI form for compute offering creation is provided with a new field named 'compute offering type’ with values - Fixed, Custom Constrained, Custom Constrained. It will allow the creation of compute offerings either with a fixed CPU speed and memory for fixed compute offering, or with a range of CPU number and memory for custom constrained compute offering or without predefined CPU number, CPU speed and memory for custom unconstrained compute offering.

To allow the user to set CPU number, CPU speed and memory during VM deployment, UI form for VM deployment has been modified to provide controls to change these values. These controls are depicted in screenshots below for custom constrained and custom unconstrained compute offering types.

Sample API calls using cmk to create a constrained service offering and deploying a VM using it,

create serviceoffering name=Constrained displaytext=Constrained customized=true mincpunumber=2 maxcpunumber=4 cpuspeed=400 minmemory=256 maxmemory=1024

deploy virtualmachine displayname=ConstrainedVM serviceofferingid=60f3e500-6559-40b2-9a61-2192891c2bd6 templateid=8e0f4a3e-601b-11e9-9df4-a0afbd4a2d60 zoneid=9612a0c6-ed28-4fae-9a48-6eb207af29e3 details[0].cpuNumber=3 details[0].memory=800

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-05-23 11:47:53 +05:30
Rohit Yadav
0700d91a68 Merge branch '4.12'
- Fixes PR #3146 db cleanup to the correct 4.12->4.13 upgrade path
- Fixes failing unit test due to jdk specific changes after forward
  merging

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-14 15:15:17 +05:30
Frank Maximus
e11f7ee1ba RIP Nuage Cloudstack Plugin (#3146)
may it rest in peaces
2019-05-14 10:58:24 +02:00
Rohit Yadav
00ff536f81 Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-14 14:26:11 +05:30
Rohit Yadav
67160478a6
server: allow admins to blacklist vm details that users should not see (#3213)
This introduces a new global setting `user.vm.blacklisted.details` that
allows admins to blacklist VM details that non-admin users should not
see via the VM's settings tab.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-10 23:39:13 +05:30
Rohit Yadav
bfad334117
api: include tags in listvmsnapshots response (#3216)
This ensures that tags of a VM snapshot are listed in the UI, available
in the list vmsnapshots API response.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-05-10 23:37:18 +05:30
Gerd Müller
de545229f9 Include 'removed' async jobs to check recurring snapshots (#3190) 2019-05-01 16:49:09 +02:00
Abhishek Kumar
6dad46b7ed server: sync templates on adding new secondary storage (#3302)
Fixes #2689

With the current code, existing templates were not downloaded to the new secondary storage when it is added. SSVM needed to be restarted to start the download process. This PR starts templates sync for the new secondary storage when it is added.

Signed-off-by: Abhishek Kumar <abhishek.kumar@shapeblue.com>
2019-04-29 23:37:15 +05:30
Gabriel Beims Bräscher
d6437d5aca server: use latest InfluxDB (2.15), add support to Batch Mode (#3289)
The InfluxDB Java client supports Batch Mode at versions 2.9+ [1]. Thus, this PR updated to the latest InfluxDB (2.15), adding support to Batch Mode

[1] https://github.com/influxdata/influxdb-java
2019-04-29 23:35:37 +05:30
Abhishek Kumar
6bb8d977ac ui: show complete domain for accounts (#2994) (#3256)
This allows showing complete domain, ie, domain path for accounts list view and account detail.
Added a new key, domainpath, in AccountResponse.

Fixes #2994

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-04-10 17:18:01 +05:30
Abhishek Kumar
9b14e17a58 server: fix for vm snapshot search (#3208) (#3257)
Using db column instead of VO variable name was causing issue with SQL select statement.
This change fixes the problem by using VO variable for adding conditional.

Fixes #3208

Using db column instead of VO variable name was causing issue with SQL select statement.
This PR fixes the problem by using VO variable for adding conditional.
Additionally in UI listAll parameter was being sent twice in the listVMSnapshot API call. It is fixed with this PR.

Signed-off-by: Abhishek Kumar <abhishek.mrt22@gmail.com>
2019-04-08 14:18:00 +05:30
Daan Hoogland
0237ee7cd5 Merge release branch 4.12 to master
* 4.12:
  fix bugs that may cause program crash, change mkdir to mkdirs (#3249)
2019-04-05 11:32:57 +02:00
bd2019us
09045bd3e5 fix bugs that may cause program crash, change mkdir to mkdirs (#3249)
Fixes: [#CLOUDSTACK-10406](https://issues.apache.org/jira/browse/CLOUDSTACK-10406)
2019-04-05 11:14:04 +02:00
skattoju3
beb0422dd5 fix incorrect iscsi path stat for managed storage (#3181)
## Description
The issue was that an Incorrect iscsi path was being passed for managed storage pools when collecting volume stats. Storage pools normally have a UUID based path while managed storage pools require an IQN based path
2019-03-29 10:25:01 +01:00
Rohit Yadav
462a37d1b4
server: make snapshotting on KVM non-blocking (#3209)
* server: make snapshotting on KVM non-blocking

This references and uses an already fixed solution from
https://github.com/MissionCriticalCloud/cosmic/pull/68 to make
snapshotting on KVM non-blocking.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* move StorageSubSystemCommand instanceof check above as CopyCommand is a type of StorageSubSystemCommand

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-03-25 13:57:21 +05:30
Rohit Yadav
a3938b330a Merge remote-tracking branch 'origin/4.12' 2019-03-25 13:27:28 +05:30
Rohit Yadav
bbf5807688 Merge remote-tracking branch 'origin/4.11' into 4.12
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2019-03-25 13:24:46 +05:30
Dingane Hlaluku
0d21f0c1e7 api: rename ListUsageRecords file name to ListUsageRecordsCmd (#3151)
* Rename ListUsageRecords API command file name to ListUsageRecordsCmd

* Refactor to use APINAME variable and remove unused s_logger field

* Remove unused import
2019-03-25 13:10:53 +05:30
Gabriel Beims Bräscher
7c5eca9481
Copy template to target KVM host if needed when migrating local <> local storage (#3154)
* Migrate template to target host if needed.

Fix KVM VM local storage live migration by migrating its template to the
target host if needed.

* Address reviewer and add method that updates the DB template reference

* Remove deprecated Config.PrimaryStorageDownloadWait

* Code formating of @Inject to follow checkstyle
2019-02-05 00:18:29 -02:00
Nathan Johnson
637cc6ec4e feature: add libvirt / qemu io bursting (#3133)
* feature: add libvirt / qemu io bursting

Adds the ability to set bursting features from libvirt / qemu

This allows you to utilize the iops and bytes temporary "burst" mode
introduced with libvirt 2.4 and improved upon with libvirt 2.6.

https://blogs.igalia.com/berto/2016/05/24/io-bursts-with-qemu-2-6/

* updates per rafael et al
2019-02-04 19:47:44 -02:00
GabrielBrascher
460d3127ec Fix conflict and merge forward PR #3122 from 4.11 to master (4.12) 2019-02-04 19:24:59 -02:00
dahn
58466c8954 api: add command to list management servers (#3150)
* api: add command to list management servers

* api: add number of mangement servers in listInfrastructure command

* ui: add block for mangement servers on infra page

* api name resolution method cleanup
2019-02-01 16:23:39 -02:00
Wido den Hollander
9c4292cc45 network: Offerings do not have to have Security Grouping enabled (#3112)
Offerings can co-exist where on does provide Security Grouping in the
network, but other guest Networks have no Security Grouping.

In V(X)LAN isolation environments the L2 separation is handled by V(X)LAN
and protection between Instances is handled by Security Grouping.

There are multiple scenarios possible where one network has Security Grouping
enabled because that is required in that network.

In the other network, but in the same zone it could be a choice to have
Security Grouping disabled and allow all traffic to flow.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2019-01-28 15:38:08 -02:00
dahn
b363fd49f7 Vmware offline migration (#2848)
* - Offline VM and Volume migration on Vmware hypervisor hosts
- Also add VM disk consolidation call on successful VM migrations

* Fix indentation of marvin test file and reformat against PEP8

* * Fix few comment typos
* Refactor debug messages to use String.format() when debug log level is enabled.

* Send list of commands returned by hypervisor Guru instead of explicitly selecting the first one

* Fix unhandled NPE during VM migration

* Revert back to distinct event descriptions for VM to host or storage pool migration

* Reformat test_primary_storage file against PEP-8 and Remove unused imports

* Revert back the deprecation messages in the custom StringUtils class to favour the use of the ApacheUtils
2019-01-25 10:05:13 -02:00
Bitworks LLC
d68712eb7b CLOUDSTACK-3049: Implemented role update for account. (#3058) 2019-01-25 10:02:56 -02:00
Malcolm Taylor
97ddd8dffd Fix some LGTM alerts (#3143)
* Fix some LGTM alerts

* address review comments from Gabriel Brascher and Rafael Weingartner
2019-01-24 20:22:39 -02:00
Dingane Hlaluku
323f791efc IP address acquired with associate ip address is marked as source nat (#3125)
* CLOUDSTACK-4045 added a check for network state when determining whether a new IP should be source NAT. this prevents associated IP's to be marked as source NAT when the network is in allocated state, causing disassociateIpAddress to fail later

* Remove mock object that cause other tests to fail

* Remove underscores from variable types and add documentation for the created method

* Improve exception message to include network name

* Include network UUID with the Exception message and fix failing marvin test

* Rebase against latest master and format AssociateIPAddrCmd class
2019-01-23 10:05:16 -02:00
Wido den Hollander
f967944d90 ipv6: Do not allow Secondary IPv6 addresses to be EUI-64 (#3136)
* netutils: Add method to verify if IPv6 Address is EUI-64

By checking if ff:fe is present in the address we can see if an IPv6 Address
is EUI-64 or not.

Signed-off-by: Wido den Hollander <wido@widodh.nl>

* ipv6: Do not allow a Secondary IPv6 address to be EUI-64

EUI-64 addresses should not be allowed as they can be used in the future by a to be
deployed Instance which has to obtain this address because it matches it's MAC.

In a /64 subnet there are more then enough other IPs available to be allocated to
Instances, therefor we can safely disallow the allocation of EUI-64 addresses.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2019-01-21 09:20:27 -02:00
Wido den Hollander
af9fefc6b4 ipv6: Calculate IPv6 address instead of fetching one from a pool (#3077)
With IPv6 we are not using DHCP to allocate addresses, but using
StateLess Address Auto Configuration (SLAAC) a Instance will calculate
it's own address based on the Router Advertisements send out by the
routers in the network.

This Advertisement contains the IPv6 Subnet in use in that subnet and
allows to calculate the stable Address the Instance will obtain based
on it's MAC Address.

The existing code is 'dead code' as it has been written, but was never
used by any production code.

SLAAC only works properly with subnets of exactly 64-bits large.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2019-01-16 16:49:02 -02:00
Anurag Awasthi
1a6eb4b856 server: use resource UUID instead of resource ID in API response (#2527) (#3099)
List resourcedetails was adding DB ID instead of UUID. This is a security risk and needs fix.

Fixes: #2527
2019-01-15 14:05:16 +05:30
Rafael Weingärtner
1725130975 server: Fix failing test_nic_secondaryip_add_remove test (#3129)
This regression was introduced with PR #2773 (Support IPv6 address in addIpToNic). The contributor did not take into consideration that the method “addIpToNic” was designed to add/allocate other IPs to a NIC. If users did not specify an IP, ACS should generate one for the network where the NIC is plugged into.

Even though I am fixing this regression here, it is still important to highlight that for IPV6, the user is not able to allocate an IP without specifying it.
2019-01-15 13:50:29 +05:30
Gabriel Beims Bräscher
1e0be522b1
Add influxdb to statscollector (#3078)
* Add Support for InfluxDB on StatsCollector

* Code refactored to fit Inner Class architecture.

Due to the inner class structure, test case for some methods will not be
implemented. On the future it will be necessary to refactor the whole
StatsCOllector architecture and extract inner classes.

Each Inner Class that is a "stats collector" and sends data to Influx
will extend AbstractStatsCollector to send metrics to the correct
measure ("table"). For instance, HostCollector sends data to host_stats,
VmStatsCollector sends data to vm_stats.

Add ping test for ensure that the target InfluxDB host is reachable

* Address PR reviews

* Enhance and tests implemented addressing reviewers.

* Set variables to private
2019-01-09 11:22:35 +01:00
Rohit Yadav
92cc4514ea Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-12-28 15:20:23 +05:30
Rohit Yadav
3424d9e6ff Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-12-28 15:11:15 +05:30
Anurag Awasthi
3279707030 api: Throw InvalidParameterValueException for failing ApiArgValidator (#3108)
This is important because it helps in communicating back the exact
error to the API callee.

Current behavior is that ParamProcessWorker#processParameters catches
the exception and returns an incorrect type exception without the
proper message.
2018-12-24 23:50:55 +05:30
Wido den Hollander
2699586d92 network: Send userdata to Virtual Router if IPv6 is enabled (#3100)
There is no reason to not send userdata+password to the VR as all
Instances in CloudStack are Dual-Stacked. They have IPv4 and IPv6
so they can query their metadata over IPv4 at the VR.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2018-12-24 23:42:57 +05:30
Gabriel Beims Bräscher
bf209405e7 Allow KVM VM live migration with ROOT volume on file storage type (#2997)
* Allow KVM VM live migration with ROOT volume on file

* Allow KVM VM live migration with ROOT volume on file
- Add JUnit tests

* Address reviewers and change some variable names to ease future
implementation (developers can easily guess the name and use
autocomplete)
2018-12-14 09:01:28 -02:00
Henko
525ddfb717 Destroyvm also removes volumes (#2793)
* Allow user to detach and delete volumes when destroyinh VMs

* Minor code refactoring
2018-11-30 10:27:31 -03:00
Anurag Awasthi
6a4a4daa46 api: Allow password enabled for iso (#2745) (#3042)
Both register and update iso commands can allow for allowing password
enabled.
Fixes: #2745

Fixe: #2745
2018-11-28 00:09:20 +05:30
Gabriel Beims Bräscher
2334145602
Support requesting a specific IPv4 address (#2595)
This commit allows deploying VMs with a specific IPv4 address.

DirectPodBasedNetworkGuru does not support requesting a custom
IP-Address while creating a new NIC/Instance, throwing the following
error:

    Error 530: Does not support custom ip allocation at this time:
NicProfile[0-0-null-null-null
    Unknown macro: { "cserrorcode"}

Some use-cases prefer the ability to request the IPv4 address which the
Instance will get.

This implementation adds unit test cases to cover and it was manually
tested in Basic Networking. I can perform more tests if requested.
2018-11-27 09:39:10 -02:00
Wido den Hollander
c499be256a ipv6: Advanced Networking Security Groups are supported (#3053)
With earlier work in Basic Networking and the security group provider IPv6 is
supported and we can allow IPv6 to be supplied in networks with SG enabled.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2018-11-24 15:10:36 +05:30
Rohit Yadav
375264aaa4 Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-11-12 12:12:08 +05:30
Wido den Hollander
44c080da11 server: print log on INFO if Host reached Max Guests Limit (#3013)
This should not be in DEBUG as people would want to know that the host was skipped
because it didn't have enough slots available to run the VM.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2018-11-12 11:39:17 +05:30
Rafael Weingärtner
f18326880b Forwardmerge #2980 - Fix set initial reservation on public IP ranges 2018-11-07 10:50:09 -02:00
Nicolas Vazquez
4de4eabd18
Enable DPDK support on KVM (#2839)
* Enable DPDK support on KVM

* Allow DPDK deployments on user VMs only

* Fix port name ordering
2018-11-07 09:29:01 -03:00
Rafael Weingärtner
077e692dbb Forward merge Fix DirectNetworkGuru canHandle for lowercase isolation methods
forward merge of #3010 -- Fix DirectNetworkGuru canHandle checks for lowercase isolation methods
2018-11-07 09:56:05 -02:00
Yoan Blanc
17c164d59a api: signature v3 to accept more formats (#2893)
It does it by reusing the DateUtil helpers. DateUtil uses java.time.* as that one knows how to deal
with timezones correctly.

The format expected by signatureVersion=3&expires=.... is quite limited.

It should accept the following formats that are containing a timezone and/or milliseconds.

2018-10-01T08:12:14Z
2018-10-01T08:12:14+01:00
2018-10-01T08:12:14+0100
2018-10-01T08:12:14.000Z
2018-10-01T08:12:14.000+01:00
2018-10-01T08:12:14.000+0100
afaik only 2018-10-01T08:12:14+0100 is accepted by the current codebase.

This PR echoes other pull requests I made earlier this year. #2392 and #2867

Signed-off-by: Yoan Blanc <yoan.blanc@exoscale.ch>
2018-10-31 16:57:48 +05:30
Gabriel Beims Bräscher
cdc6e6e50a CLOUDSTACK-9315: Removed unused Classes (#1448)
* Remove some unused Classes

These classes were deleted because they have no references in our code base. They are not in Spring execution flow nor instantiated with "new":
- com.cloud.agent.api.CheckStateAnswer
- com.cloud.agent.api.StartupVMMAgentCommand
- com.cloud.agent.api.routing.UserDataCommand
	- remove from description at
com.cloud.configuration.Config.ExecuteInSequenceNetworkElementCommands
enum
- com.cloud.agent.api.storage.UpgradeDiskCommand
- com.cloud.agent.api.storage.CreatePrivateTemplateCommand
- com.cloud.agent.api.storage.DestroyAnswer
	- Note: "FIXME: Should have an DestroyAnswer" at
com.cloud.storage.resource.StoragePoolResource
- com.cloud.agent.api.storage.UpgradeDiskAnswer
- com.cloud.agent.api.storage.ManageVolumeAvailabilityAnswer
- com.cloud.agent.api.storage.ManageVolumeAvailabilityCommand
- com.cloud.exception.UsageServerException
- com.cloud.info.SecStorageVmLoadInfo
- com.cloud.serializer.SerializerHelper

* PR#1448 update description of 'execute.in.sequence.network.element.commands' param

Update description of 'execute.in.sequence.network.element.commands'parameter to reflect an unused command that has been removed. The removed class command is 'UserDataCommand'.

* Add cloud schema to update SQL
2018-10-29 08:27:00 -03:00
Rohit Yadav
323d381767 Merge remote-tracking branch 'origin/4.11'
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2018-10-29 16:27:08 +05:30
Abhishek
6f44a8d7ba template: Return requireshvm key as part of list template response (#2921)
Added requireshvm key in the response of list template API.
Fixes #2821

Signed-off-by: Abhishek Kumar <abhishek.mrt@gmail.com>
2018-10-24 16:20:27 +05:30