Commit Graph

1028 Commits

Author SHA1 Message Date
Rohit Yadav
0154da6417 CLOUDSTACK-9348: Unit test to demonstrate denial of service attack
The NioConnection uses blocking handlers for various events such as connect,
accept, read, write. In case a client connects NioServer (used by
agent mgr to service agents on port 8250) but fails to participate in SSL
handshake or just sits idle, this would block the main IO/selector loop in
NioConnection. Such a client could be either malicious or aggresive.

This unit test demonstrates such a malicious client that can perform a
denial-of-service attack on NioServer that blocks it to serve any other client.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-05-03 08:01:46 +05:30
Will Stevens
a6bae2b9c2 Merge pull request #1498 from GabrielBrascher/lrg-cs-hackday-038
CLOUDSTACK-9352: Test fails in Widows as the file separator "/" is different from "\"**Problem:**
File separator in windows ("\") is different from the expected in the test ("/"); thus, the test *com.cloud.utils.SwiftUtilTest.testSplitSwiftPath()* will fail in Windows systems.

The problem is that the input of the test is "*container/object*" but the tested method uses the *File.separator* (that depends from the OS), in windows systems the tested method (*com.cloud.utils.SwiftUtil.splitSwiftPath(String)*) looks for a "\", as the string does not contain "\" it returns an empty string and consequently results in a test failure.

**Solution:**
Create a string `String input = "container" + File.separator + "object";`, with that the test will validate the tested method verifying if the method splits the string around matches of the given regular expression (in this case *File.separator*).

*JIRA link: https://issues.apache.org/jira/browse/CLOUDSTACK-9352*

* pr/1498:
  Test fails in Widows as the file separator "/" is different from "\"

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-25 15:58:17 -04:00
Will Stevens
d518b619dd Merge pull request #1459 from GabrielBrascher/CLOUDSTACK-8611
This closes #561

CLOUDSTACK-8611:Handle SSH if server "forget" to send exit statusContinuing the work started by @likitha, I did not cherry-picked the
commit (b9181c689e) from PR <https://github.com/apache/cloudstack/pull/561> due to the fact that the path of that SshHelper class was different of the current SshHelper; that is because the fact that by cherry-picking it would seem that I had changed all the class as the code is from another file.

I made some changes from the cherry-picked commit adding @wilderrodrigues suggestions (create simple methods to have reusable code, make unit tests and create the `WAITING_OPEN_SSH_SESSION` variable to manipulate with the delay of 1000 milliseconds).

Also, I tried to simplify the logic by assuming that ....

    if ((conditions & ChannelCondition.EXIT_STATUS) != 0) {
            if ((conditions & (ChannelCondition.STDOUT_DATA | ChannelCondition.STDERR_DATA)) == 0) {
                break;
            }
    }

... is the same as `((conditions & ChannelCondition.EXIT_STATUS) != 0) && ((conditions & (ChannelCondition.STDOUT_DATA | ChannelCondition.STDERR_DATA)) == 0)`. This expression has the following results according to each possible condition.

|Condition|Value|result
|-----------------|-------|------|
TIMEOUT  | 0000001|false
CLOSED  | 0000010 |false
STDERR_DATA | 0000100 | false
STDERR_DATA | 0001000 | false
EOF         | 0010000 | false
EXIT_STATUS | 0100000 | **true**
EXIT_SIGNAL | 1000000 | false

After testing all the possibilities we can note that the condition of `(conditions & ChannelCondition.EXIT_STATUS) != 0` is sufficient; thus, the simplified "if" conditional can be:

`if ((conditions & ChannelCondition.EXIT_STATUS) != 0) {
    break;
}`

This proposed work can be explained by quoting @likitha:
>CheckS2SVpnConnectionsCommand execution involves executing a script (checkbatchs2svpn.sh) in the virtual router. Once CS has opened a session to a virtual router and executed a script in the router, it waits indefinitely till the session either times out or the exit status of the remote process is available. But it is possible that an EOF is reached by the process in the router and the router never set the exit status.

>References -
>1. Some servers never send the exit status, or occasionally "forget" to do so (http://grepcode.com/file/repo1.maven.org/maven2/org.jvnet.hudson/trilead-ssh2/build212-hudson-1/com/trilead/ssh2/ChannelCondition.java).
>2. Get the exit code/status from the remote command - if available. Be careful - not all server implementations return this value - (http://grepcode.com/file/repo1.maven.org/maven2/org.jvnet.hudson/trilead-ssh2/build212-hudson-1/com/trilead/ssh2/Session.java#Session.waitForCondition%28int%2Clong%29).

* pr/1459:
  Handle SSH if server "forget" to send exit status

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-21 16:59:16 -04:00
gabrascher
4be5c2e56c Test fails in Widows as the file separator "/" is different from "\"
File separator in windows is different from linux (the expected in the
test); thus, the test
*com.cloud.utils.SwiftUtilTest.testSplitSwiftPath()* will fail in
windows. The problem is that the input of the test is
*"container/object"* but the tested method uses the *File.separator*
(that depends from the OS), in the windows the tested method
(*com.cloud.utils.SwiftUtil.splitSwiftPath(String)*) looks for a "\" in
windows systems, resulting in an empty string and consequently a failure
in the test.

Some solutions:
- the simple way is to create a string `String input = "container" +
File.separator + "object";`, thus independent of the OS, the test will
succeed.
- a tricky solution is to mock the final static variable
*File.separator* and return "/".

I picked the easy way.
2016-04-21 17:55:59 -03:00
gabrascher
abae908674 Handle SSH if server "forget" to send exit status
Continued the work started by https://github.com/likitha
commit (b9181c6) from PR #561.

CS waits indefinitely for CheckS2SVpnConnectionsComm and to return.
While remote executing commands through ssh, handle channel condition of
EOF because we wait for the the condition.

The SshHelper of the PR #561 is of another path from the
current master, its path was
https://github.com/likitha/cloudstack/commits/CLOUDSTACK-8611/utils/src/com/cloud/utils/ssh/SshHelper.java;
thus, although this commit brings changes from PR #561, I did not
cherry-picked to keep the master file, otherwise it would look that I
had changed all the file.
by me.
2016-04-20 13:51:17 -03:00
Rohit Yadav
8e4644e413 vmware: improve support for disks
- Improve disk chain usage while attaching, migrating disks
- Gets root disk controller based diskDeviceBusName from volume's chain info
- Refactor and move VirtualMachineDiskInfo to cloud-utils
- Allows mixing of scsi controller types
- Fixes a NPE case with map passed as null, for example in case of detach volume
  command
- Use a osdefault translator that allow use of recent os types added (enums of
  which) are not available in the sdk

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2016-04-18 22:44:18 +05:30
Will Stevens
749b71601c Merge pull request #1445 from rafaelweingartner/lrg-cs-hackday-025
Fixed Profiler's unit tests bugs.### **Problem:**
The TestProfiler class was using Java Thread methods to test the
Profiler's functionality. That was causing the tests to fail sometimes
since the JVM's thread priority could be low on some OS.

### **Fix:**
Using PowerMockito to mock the System calls, the threads could be
removed. This makes the tests considerably faster, OS independent and
still guarantees the correct implementation of the Profiler class.

The changes on the Profiler's class was only to shorten the class's line
size by not assigning the return value to a variable returning it
straight out.

* pr/1445:
  Fixed changes to match code conventions
  Fixed Profiler's unit tests bugs.

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-18 08:30:46 -04:00
weingartner
4179606c0b Removed classes with no reference
- org.apache.cloudstack.framework.jobs.JobCancellationException (removed
variable JobCancellationException in com.cloud.utils.SerialVersionUID)
- org.apache.cloudstack.ldap.NoSuchLdapUserException (removed test file
/cloud-plugin-user-authenticator-ldap/test/groovy/org/apache/cloudstack/ldap/NoSuchLdapUserExceptionSpec.groovy)
- com.cloud.agent.api.storage.CreateVolumeOVAAnswer
- com.cloud.exception.MissingParameterValueException
- org.apache.cloudstack.api.response.StatusResponse
- org.apache.cloudstack.api.response.VolumeDetailResponse
- org.apache.cloudstack.api.response.UpgradeVmResponse
- org.apache.cloudstack.api.response.AddIpToVmNicResponse
- org.apache.cloudstack.api.response.TemplateZoneResponse (at
org.apache.cloudstack.api.response.TemplateResponse, there is this
comment "To avoid breaking backwards compatibility, we still treat a
template at different zones as different templates, so not embedding
template_zone information in this TemplateZoneResponse set. `private
Set<TemplateZoneResponse> zones;`" but right now it is not used)
- org.apache.cloudstack.api.response.NicDetailResponse
2016-04-07 21:03:05 -03:00
Will Stevens
bb82f9a7fa Merge release branch 4.7 to 4.8
* 4.7:
  Add ability to download templates in Swift
2016-04-06 14:32:08 -04:00
Will Stevens
ef115ab1d0 Merge pull request #1332 from syed/swift-download
Add ability to download templates in SwiftThis PR adds the ability to download templates when using Swift as a secondary storage. Uses the "temp_url" feature of Swift so that tempates can be downloaded without authenticaiton.

* pr/1332:
  Add ability to download templates in Swift

Signed-off-by: Will Stevens <williamstevens@gmail.com>
2016-04-06 14:19:44 -04:00
Syed
52fc2bab69 Add ability to download templates in Swift 2016-04-05 16:40:00 -04:00
Alexandre Limas Santana
c2afa584ae Fixed changes to match code conventions 2016-03-20 11:49:04 -03:00
Alexandre Limas Santana
8d6370b21f Fixed Profiler's unit tests bugs.
Problem:
The TestProfiler class was using Java Thread methods to test the
Profiler's functionality. That was causing the tests to fail sometimes
since the JVM's thread priority could be low on some OS. 

Fix:
Using PowerMockito to mock the System calls, the threads could be
removed. This makes the tests considerably faster, OS independent and
still guarantees the correct implementation of the Profiler class.

The changes on the Profiler's class was only to shorten the class's line
size by not assigning the return value to a variable returning it
straight out.
2016-03-19 16:35:25 -03:00
Remi Bergsma
75b68c6829 Merge release branch 4.7 to master
* 4.7:
  Fix unable to setup more than one Site2Site VPN Connection
  FIX S2S VPN rVPC: Check only redundant routers in state MASTER
  PEP8 of integration/smoke/test_vpc_vpn
  Add S2S VPN test for Redundant VPC
  Make integration/smoke/test_vpc_vpn Hypervisor independant
  FIX VPN: non-working ipsec commands
  [UI] MADNESS
  [DB] Add force_encap field to s2s_customer_gateway table
  [ROUTER] Add forceencaps field to python router ipsec config method
  [TEST] unittest needs rework
  [MARVIN] Add forceencap field to VpnCustomerGateway class in marvin base
  [CORE] Add Force UDP Encapsulation option to Site2Site VPN
  CLOUDSTACK-9186: Root admin cannot see VPC created by Domain admin user
  CLOUDSTACK-9192: UpdateVpnCustomerGateway is failing
  CLOUDSTACK-6485 prevent ip asignment of private gw iface
  CLOUDSTACK-9204 Do not error when staticroute is already gone
  make both check lines consistent
  CLOUDSTACK-9181 Prevent syntax error in checkrouter.sh
  CLOUDSTACK-9202 Bump ssh timeout
2016-01-16 19:54:41 +01:00
Remi Bergsma
4014169ccb Merge pull request #1324 from bheuvel/testprofiler-second
Factor in inaccurancy of System.nanoTime in class ProfilerThe time diff measurement by the class Profiler, is done by System.nanoTime().
System.nanoTime() is not 100% accurate (lots of info on this on the web). The inaccurance seems to vary/depend with different dependencies (multi-core/OS/kernels).

Problem:
In my case the code which measures `Thread.sleep(ONE_SECOND);` ~~takes~~ measures 999 milliseconds, which breaks the test expecting the sleep of 1000 milliseconds to ~~take~~ measure at least 1000 milliseconds.

Solution:
A `MARGIN` (of 100 milliseconds) is already present to factor in some time for overhead/surrounding code which adds time. Another margin could be added to factor in the possible time measurement error.
To limit additional lines of code, the constant `ONE_SECOND`, which is used by the testcase (sleep), is increased from 1000 milliseconds to `1000l + (MARGIN / 10)` (effectively 1010 milliseconds).

Thereby the inaccurancy on my system of ~1% ("faster") is caught as the `sleep(1010)` is measured as '1009' milliseconds.

* pr/1324:
  Factor in inaccurancy of System.nanoTime

Signed-off-by: Remi Bergsma <github@remi.nl>
2016-01-11 16:20:51 +01:00
Bob
dbb10a4ed5 Factor in inaccurancy of System.nanoTime 2016-01-08 13:31:27 +01:00
Remi Bergsma
1c5e899a2d CLOUDSTACK-9202 Bump ssh timeout
It seems the VR needs more time for some of its commands. Until we figured
out the root cause, this allows the VRs to start again.
2015-12-28 15:54:26 +01:00
nvazquez
eb889c0c49 CLOUDSTACK-9074: API add Gateway Service Find method 2015-12-24 16:07:06 -03:00
nvazquez
8149081658 CLOUDSTACK-9074: API Changes: Add nsxLogicalSwitch and nsxLogicalSwitchPort to listNics 2015-12-24 16:07:06 -03:00
Remi Bergsma
66d7f413e8 Merge pull request #1188 from kansal/CLOUDSTACK-9086
CLOUDSTACK-9086: ACS allows to create isolated networks with invalide gateway IP address - Fixed and Test cases added

Problem: There was no check for the network and broadcast IP addresses in the case where we provide the gateway and netmask while creating the isolated network. As a result the provided IP gets assigned to the eth0 interface of the VR.

Note: This is in continuation of PR #1125 which I closed.

* pr/1188:
  CLOUDSTACK-9086: ACS allows to create isolated networks with invalide gateway IP address - Fixed and Test cases added

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-12-12 09:37:06 +01:00
Remi Bergsma
312b9aff94 Merge release branch 4.6 to master
* 4.6:
  Adapted HypervisorUtilsTest to no longer give false positives
2015-12-10 16:24:36 +01:00
Kshitij Kansal
c7c389d61c CLOUDSTACK-9086: ACS allows to create isolated networks with invalide gateway IP address - Fixed and Test cases added 2015-12-08 15:27:51 +05:30
Nick Livens
132c35639b Adapted HypervisorUtilsTest to no longer give false positives 2015-12-08 08:12:58 +01:00
Daan Hoogland
afe1130920 Merge pull request #1057 from DaanHoogland/CWE-190
Cwe 190coverity warnings of this type adressed. Some where dismissed and maybe with reason but it seemed possible to remove them and hence obligatory ;p

* pr/1057:
  move back to original contract of isNetworksOverlap()
  Changed the behavior of methods that use NetUtils.cidrToLong(String)
  CWE-190 unit test for extremes of long netMaskFromCidr(long)
  CWE-190 netmask as long form cidr-size as method
  CID-1116482 cidrToLong cleanup of bitshift problem
  CID-1116483 cidr to netmask bitshifts guarded with casts
  CID-1116484 cast to long and use long as cidrsize type  and simpel test
  CID-1116485: cast cidr during bit shifting  and simple test included
  CID-1175714 casts before bit shift

Signed-off-by: Daan Hoogland <daan@onecht.net>
2015-12-06 20:10:57 +01:00
Remi Bergsma
a6b6f4104d Merge pull request #1102 from rafaelweingartner/master-lrg-cs-hackday-006
Removed unnecessary @Local annotations and their respective importsFollowing @rafaelweingartner 's findings in PR #714 that many spring beans contained an @Local annotation, we've decided to remove said annotations and their imports from the ComponentLifecycleBase class and its subclasses seeking a reduction of a few hundred lines of useless code.

I had already opened a pull request for this (#853) but at some point my commit disappeared from the PR entirely, showing no new changes in code, which caused it to be merged automatically (with no changes).

* pr/1102:
  Removed unnecessary @Local annotations and their respective imports from the ComponentLifecycleBase class and its subclasses.

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-11-23 11:25:43 +01:00
Remi Bergsma
a3688e0eb5 Merge release branch 4.6 to master
* 4.6:
  CLOUDSTACK-9052 Shuffling the password to avoid having a subset of characters in fixed positions.
  Refactor package.sh:  * lint  * adjust exit codes (1 for usage, 2 for maven, 3 for rpmbuild)  * variable naming consistency  * add option for package release version  * revise synopsis and usage
2015-11-22 12:52:43 +01:00
cirstofolini
1a64c247ad Removed unnecessary @Local annotations and their respective imports from the ComponentLifecycleBase class and its subclasses. 2015-11-21 18:31:11 -02:00
Daan Hoogland
4b7f87b9c8 move back to original contract of isNetworksOverlap() 2015-11-21 18:58:54 +01:00
weingartner
1d6ee2dcb0 Changed the behavior of methods that use NetUtils.cidrToLong(String)
Given that the method com.cloud.utils.net.NetUtils.cidrToLong(String)
now throws an exception when receiving null or empty cidrs, there is the
need to change methods that use it. Those methods were changed and test
cases created.
2015-11-20 13:31:58 -02:00
Daan Hoogland
02058b939c CWE-190 unit test for extremes of long netMaskFromCidr(long) 2015-11-19 19:37:26 +01:00
Boris Schrijver
5c0366c99e CLOUDSTACK-9062: Improve S3 implementation.
The S3 implementation is far from finished, this commit focusses on the bases.

 - Upgrade AWS SDK to latest version.
 - Rewrite S3 Template downloader.
 - Rewrite S3Utils utility class.
 - Improve addImageStoreS3 API command.
 - Split various classes for convenience.
 - Various minor improvements and code optimalisations.

A side effect of the new AWS SDK is that it, by default, uses the V4 signature. Therefore I added an option to specify the Signer, so it stays compatible with previous versions.
2015-11-19 15:29:10 +01:00
nnesic
52ccfaac0c CLOUDSTACK-9052 Shuffling the password to avoid having a subset of characters in fixed positions. 2015-11-19 13:17:20 +00:00
nvazquez
b20c5be1ce CLOUDSTACK-8956: Remove assert(false) on opaque network and ping method on NiciraNvpApiVersion 2015-11-16 11:29:31 -08:00
nvazquez
2386ae5e5a CLOUDSTACK-8956: Log NSX Api Version 2015-11-16 11:29:30 -08:00
Daan Hoogland
1bc837837f CWE-190 netmask as long form cidr-size as method 2015-11-16 18:25:32 +01:00
Daan Hoogland
3ae4dd06f3 CID-1116482 cidrToLong cleanup of bitshift problem
and some trivial testing
2015-11-16 18:25:32 +01:00
Daan Hoogland
f9d5c6918c CID-1116483 cidr to netmask bitshifts guarded with casts
and simple test
2015-11-16 18:25:32 +01:00
Daan Hoogland
2fadfe93dc CID-1116484 cast to long and use long as cidrsize type
and simpel test
2015-11-16 18:25:32 +01:00
Daan Hoogland
637afb9b67 CID-1116485: cast cidr during bit shifting
and simple test included
2015-11-16 18:25:31 +01:00
Daan Hoogland
a580330864 CLOUDSTACK-9023 filtering privatekey info from logging 2015-11-10 12:08:01 +01:00
Wido den Hollander
602231132d kvm: Add UnitTests for LibvirtUtilitiesHelper
These were lacking, but this helper is used in various places
inside the KVM code.

Some simple tests to verify the helper is doing what we expect it
to do.
2015-10-30 12:34:00 +01:00
Daan Hoogland
d0a1820394 CLOUDSTACK-8981 coded a more obscure host and clear failure message
test fails when port is reachable so prevent it as much as possible
 making sure that all kinds of weird developers can work with it
2015-10-22 15:35:50 +02:00
Boris Schrijver
b34f86c8d5 Added return statement to stop start() if there has been an ConnectException. 2015-09-22 00:38:16 +02:00
wilderrodrigues
6e426fa46d Revert "Merge pull request #825 from borisroman/charset"
This reverts commit a44e2bff7e, reversing
changes made to 06cefaf493.

We are reverting the PR #825 because it breaks Master.
2015-09-15 14:58:18 +02:00
Boris Schrijver
71929fdedd Replaced all occurences of Charset.forName(UTF-8) with StringUtils.getPreferredCharset(). 2015-09-15 12:00:33 +02:00
Wido den Hollander
d091b9189c Merge pull request #795 from borisroman/CLOUDSTACK-8819
CLOUDSTACK-8819: Added QCOW2 virtual size checking for S3.- Cleaned up S3TemplateDownloader
- Created static QCOW2 utils class.
- Reformatted some parts of DownloadManagerImpl

* pr/795:
  Added Unit Tests for QCOW2Utils.
  Fixed Findbugs issue introduced by 1c6378ec00 PR #795.
  Added QCOW2 virtual size checking for S3.

Signed-off-by: Wido den Hollander <wido@widodh.nl>
2015-09-14 12:02:03 +02:00
Boris Schrijver
aa19a6a05a Added Unit Tests for QCOW2Utils. 2015-09-14 10:56:21 +02:00
Boris Schrijver
1c6378ec00 Added QCOW2 virtual size checking for S3.
- Cleaned up S3TemplateDownloader
- Created static QCOW2 utils class.
- Reformatted some parts of DownloadManagerImpl
2015-09-11 14:57:32 +02:00
wilderrodrigues
79a3f8c577 CLOUDSTACK-8822 - Replacing Runnable by Callable in the Taks and NioConnection classes
- All the sub-classes were also updated according to the changes in the super-classes
   - There were also code formatting changes
2015-09-11 11:28:40 +02:00
Boris Schrijver
1971614e31 Added QCOW2 virtual size checking for S3.
- Cleaned up S3TemplateDownloader
- Created static QCOW2 utils class.
- Reformatted some parts of DownloadManagerImpl
2015-09-10 16:45:48 +02:00
Rohit Yadav
dd9ba48efa Merge pull request #753 from shapeblue/master-CLOUDSTACK-8762
CLOUDSTACK-8762: Check to confirm disk activity before starting a VMImplements a VM volume/disk file activity checker that checks if QCOW2 file
has been changed before starting the VM. This is useful as a pessimistic
approach to save VMs that were running on faulty hosts that CloudStack could
try to launch on other hosts while the host was not cleanly fenced. This is
optional and available only if you enable the settings in agent.properties
file, on per-host basis.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

* pr/753:
  CLOUDSTACK-8762: Check to confirm disk activity before starting a VM

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-09-02 14:55:54 +05:30
Rajani Karuturi
22c78ede32 Fixing line endings in the new file introduced in PR #762 merged through commit e8979c0e65 2015-09-01 14:34:54 +05:30
Rajani Karuturi
e8979c0e65 Merge pull request #762 from rafaelweingartner/cglibThrowableRenderer
Proposal for an improved CglibThrowableRendererFollowing our discussions on mailing list, here is a PR with a proposal for an improvement on CglibThrowableRenderer. We would still have to define a number to limit the stack traces of causes (I randomly chose 3, just to get the code running). The test case is also not good in my opinion, but I ran out of ideas (waiting for suggestions on that).

* pr/762:
  Changed the test case to test each of the log traces in the array list.
  Proposal for an improved CglibThrowableRenderer

Signed-off-by: Rajani Karuturi <rajani.karuturi@citrix.com>
2015-09-01 14:31:32 +05:30
weingartner
b49d681598 Changed the test case to test each of the log traces in the array list. 2015-08-31 07:51:01 -03:00
Rajani Karuturi
8bc0294014 Revert "Merge pull request #714 from rafaelweingartner/master-lrg-cs-hackday-003"
This reverts commit cd7218e241, reversing
changes made to f5a7395cc2.

Reason for Revert:

noredist build failed with the below error:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.2:compile (default-compile) on project cloud-plugin-hypervisor-vmware: Compilation failure
[ERROR] /home/jenkins/acs/workspace/build-master-noredist/plugins/hypervisors/vmware/src/com/cloud/hypervisor/guru/VMwareGuru.java:[484,12] error: non-static variable logger cannot be referenced from a static context
[ERROR] -> [Help 1]

even the normal build is broken as reported by @koushik-das on dev list
http://markmail.org/message/nngimssuzkj5gpbz
2015-08-31 11:27:57 +05:30
weingartner
89bac529cc Proposal for an improved CglibThrowableRenderer
Apache license
Shame on me...

Fix test to pass on travis build
2015-08-29 21:49:56 -03:00
Rafael Weingartner
3818257a68 Solved jira ticket: CLOUDSTACK-8750 2015-08-28 22:35:08 -03:00
Rajani Karuturi
f5a7395cc2 Merge pull request #752 from ustcweizhou/BasicRestClientTest-fakelocalhost
use fakelocalhost in BasicRestClientTestBasicRestClientTest will fail if the port 443 is open on localhost.
use fakelocalhost instead of localhost can fix it.
Please see the latest discussion in https://github.com/apache/cloudstack/pull/737

* pr/752:
  use fakelocalhost in BasicRestClientTest

Signed-off-by: Rajani Karuturi <rajani.karuturi@citrix.com>
2015-08-28 22:12:42 +05:30
Rohit Yadav
52a98fa6cf CLOUDSTACK-8762: Check to confirm disk activity before starting a VM
Implements a VM volume/disk file activity checker that checks if QCOW2 file
has been changed before starting the VM. This is useful as a pessimistic
approach to save VMs that were running on faulty hosts that CloudStack could
try to launch on other hosts while the host was not cleanly fenced. This is
optional and available only if you enable the settings in agent.properties
file, on per-host basis.

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-28 16:05:30 +05:30
Miguel Ferreira
4c1a5f7657 Putting CglibThrowableRenderer.java back after it was removed in 83fd8f6
Also removing the entry 'log/' from .gitignore since that was the
culprit for the removal of the file.
2015-08-28 11:18:44 +02:00
Wei Zhou
5e792e33ec use fakelocalhost in BasicRestClientTest 2015-08-27 10:32:57 +02:00
wilderrodrigues
d74d134698 Removing unit test that was covering the resolution of the nano time
- The difference between 2 consecultive calls to System.nanoTime() returns a resolution that depends on the OS implementation. Due to that, the test was failing in an inttermmitent manner
   - Today we decided to remove the test since it was not testing a direct feature of ACS
2015-08-26 20:00:59 +02:00
Miguel Ferreira
09f7153378 Use NSX specific RestClient in API implementation (NiciraNvpApi)
- Simplify public API to return Lists instead of NiciraNvpLists
2015-08-25 17:36:17 +02:00
Miguel Ferreira
18e6b9ba78 Add NSX specific RestClient implementation
- Add -noverify JVM arg to surefire plugin, to allow Powermockito to
  de-encapsulate private methods
- Add dependency on cloud-utils test-jar to use custom HttpRequest
  matchers
2015-08-25 17:36:15 +02:00
Miguel Ferreira
de63b94f2d Delegate HTTP protocol activity in RESTServiceConnector to RestClient
- All HTTP protocol activities are now handled by RestClient
- This service is now only responsible for creating requests, and
  dispatching them to the client
- Provides a Simple API for creating, updating, retrieving and deleting
  objects
2015-08-25 17:36:14 +02:00
Miguel Ferreira
8a93bb8d2d Add basic RestClient implentation based on HTTP Components 4.5
- Upgrade version of HTTP Components to 4.5
- Add helper class to create Http clients
- Add helper class to build http requests
- Add enum with the different Http Methods
- Add constants class for HTTP related values
2015-08-25 17:36:13 +02:00
Miguel Ferreira
5fc953eda2 Code clean up in cloud-utils project
- Code formatting
- Remove unused methods
2015-08-25 17:36:11 +02:00
Miguel Ferreira
83fd8f60f3 Refactor cloud-utils project into Maven default structure 2015-08-25 14:50:33 +02:00
Remi Bergsma
9c7e81e748 Merge pull request #695 from karuturi/CLOUDSTACK-8669
CLOUDSTACK-8669: create volume failed due to null charsetAdded a new private method getCharSetFromConnection() which checks if
the connection charset is null and if it is null, returns
StringUtils.getPreferredCharset

regression caused by commit f03411ca04

* pr/695:
  CLOUDSTACK-8669: create volume failed due to null charset
  CLOUDSTACK-8669: making StringUtils.getDefaultCharset protected

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-08-17 13:54:54 +02:00
Remi Bergsma
64ff67da55 Merge pull request #654 from DaanHoogland/CLOUDSTACK-8656
Cloudstack 8656: do away with more silently ignoring exceptions.a lot of messages added.
some restructuring for test exception assertions and try-with-resource blocks

* pr/654: (29 commits)
  CLOUDSTACK-8656: more logging instead of sysout
  CLOUDSTACK-8656: use catch block for validation
  CLOUDSTACK-8656: class in json specified not found
  CLOUDSTACK-8656: removed unused classes
  CLOUDSTACK-8656: restructure of tests
  CLOUDSTACK-8656: reorganise sychronized block
  CLOUDSTACK-8656: restructure tests to ensure exception throwing
  CLOUDSTACK-8656: validate the throwing of ServerApiException
  CLOUDSTACK-8656: logging ignored exceptions
  CLOUDSTACK-8656: try-w-r removes need for empty catch block
  CLOUDSTACK-8656: try-w-r instead of clunckey close-except
  CLOUDSTACK-8656: deal with empty SQLException catch block by try-w-r
  CLOUDSTACK-8656: unnecessary close construct removed
  CLOUDSTACK-8656: message about timed buffer logging
  CLOUDSTACK-8656: message about invalid number from store
  CLOUDSTACK-8656: move cli test tool to separate file
  CLOUDSTACK-8656: exception is the rule for some tests
  CLOUDSTACK-8656: network related exception logging
  CLOUDSTACK-8656: reporting ignored exceptions in server
  CLOUDSTACK-8656: log in case we are on a platform not supporting UTF8
  ...

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-08-14 21:38:49 +02:00
Daan Hoogland
08e6e1633c CLOUDSTACK-8656: message about timed buffer logging 2015-08-14 13:29:35 +02:00
Daan Hoogland
4213a22127 CLOUDSTACK-8656: message about invalid number from store 2015-08-14 13:21:46 +02:00
Daan Hoogland
3229d96015 CLOUDSTACK-8656: move cli test tool to separate file 2015-08-14 13:15:11 +02:00
Rohit Yadav
1d190fd0bb utils: add HttpUtils test
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit 0fcd9cad1e)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-14 11:59:31 +05:30
Rohit Yadav
89f47ece3d CLOUDSTACK-8702: Add/refactor sessionkey checking code to HttpUtils
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
(cherry picked from commit cb7dd7b27d)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>
2015-08-14 11:56:55 +05:30
Rajani Karuturi
a65339ff1b CLOUDSTACK-8669: making StringUtils.getDefaultCharset protected
made this protected as its only use is in the Test file.
2015-08-13 17:39:44 +05:30
Daan Hoogland
c0534756d0 CLOUDSTACK-8656: network related exception logging 2015-08-11 18:14:20 +02:00
Mike Tutkowski
8b0266d12e Merge branch 'pr/547'
* pr/547:
  CLOUDSTACK-8601. VMFS storage added as local storage can be re-added as shared storage. Fail addition of a VMFS shared storage pool in case it has already been added as local storage in CS.

Signed-off-by: Mike Tutkowski <mike.tutkowski@solidfire.com>
2015-08-10 19:00:53 -06:00
Remi Bergsma
7b47c49bb0 #651 (@borisroman) CLOUDSTACK-8703: Fixed issue when listing direct..
* pr/651:
  CLOUDSTACK-8703: Fixed issue when listing directory on S3, it would only return objectSummaries when the anwser from the S3 System was truncated.

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-08-04 13:50:29 +02:00
Daan Hoogland
e8a00ed989 CLOUDSTACK-8656: try-with-resource in vmsd reader
moved closeable util function up the hierarchy
2015-08-04 11:37:06 +02:00
Remi Bergsma
60633301ac Merge pull request #647 from @wido
CLOUDSTACK-8640: Revert to AWS SDK 1.3.22

* pr/647:
  CLOUDSTACK-8640: Revert to AWS SDK 1.3.22

Signed-off-by: Remi Bergsma <github@remi.nl>
2015-08-03 20:16:04 +02:00
Boris Schrijver
5f87e9c917 CLOUDSTACK-8703: Fixed issue when listing directory on S3, it would only return objectSummaries when the anwser from the S3 System was truncated. 2015-08-03 17:10:05 +02:00
Wido den Hollander
c32656a7ef CLOUDSTACK-8640: Revert to AWS SDK 1.3.22
The newer SDKs API changed which causes our S3 Template Downloader to never complete.

Although we should fix the Template Downloader we can revert to the old AWS SDK for now.

The fix on the longer run will be rewriting the S3 Template Downloader.

Two methods had to be disabled for now since the old SDK does not support them. They can
be re-enabled when the Template Downloader has been fixed.
2015-07-31 12:52:28 +02:00
Daan Hoogland
85e002b230 CLOUDSTACK-8656: filling empty catch block with info messages
using regexp "catch\s*\(\s*(Exception|Throwable)\s*\w*\)\s*\{\s*\}"
2015-07-30 15:51:52 +02:00
Daan Hoogland
22f194b888 coverity 1125362, 1125363: assorted resource leaks
Signed-off-by: Daan Hoogland <daan.hoogland@gmail.com>

This closes #602
2015-07-27 14:43:11 +02:00
wilderrodrigues
d632d6312b CLOUDSTACK-8660 - Adding new StringUtils.getDefaultCharset() in order to wrap the Charset equivalet method.
- This test was added in order to cover the new StringUtils.getDefaultCharset().
     - One cannot be sure that StringUtils.getPreferredCharset() will always be equals to Charset.defaultCharset()

Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>

This closes #616
2015-07-23 08:08:46 +02:00
wilderrodrigues
245c976ad0 CLOUDSTACK-8660 - Adding a method to check if UTF-8 is supported
- Changing the test to call isUtf8Supported() before checking if the preferred charset is actually UTF-8

Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
2015-07-23 08:08:46 +02:00
wilderrodrigues
59d7bc3573 CLOUDSTACK-8660 - Formatting test and utility classes
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
2015-07-23 08:08:45 +02:00
Daan Hoogland
822ea9c787 CLOUDSTACK-8658: make initializer static instead of default
Signed-off-by: wilderrodrigues <wrodrigues@schubergphilis.com>
2015-07-22 10:06:31 +02:00
Wido den Hollander
178a9f555d CLOUDSTACK-8581: S3, make connection TTL and TCP KeepAlive configureable
Signed-off-by: Wido den Hollander <wido@widodh.nl>
2015-07-17 10:07:04 +02:00
Rohit Yadav
42940a8828 CLOUDSTACK-8622: Reinstate working sessions in browser
- Login is based on sessionkey HttpOnly Cookie
- ApiServlet does login verification using sessionKey from both the request cookies
  and the API parameters. In both cases, if either or both are passed they should
  match the sessionKey stored in the current session of the HttpRequest
- UI: it no longer needs to read or set sessionkey cookie
- UI: it no longer needs to return g_sessionKey value in the API requests, though
  to support a sso mechanism g_sessionKey is still passed in the API is not null
- Secure jsessionid cookie is set to be HttpOnly and Secure
- SAML login should also set HttpOnly cookie before redirecting to UI
- SAML: listIdps & getSPMetadata APIs are readonly now, won't log out a logged in user

Performed tests (login, saml login if applicable, page refreshes, opening
multiple tabs, logout) with following combinations:
- SAML disabled, normal auth as admin, domain-admin and user
- SAML enabled, normal auth as admin, domain-admin and user; and saml sso as
  admin, domain-admin and user

Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

This closes #574
This closes #308
2015-07-10 17:25:07 +05:30
Daan Hoogland
bcc8182f2f findbugs: write to static from instance these are the trivial cases of ST_WRITE_TO_STATIC_FROM_INSTANCE_METHOD as reported by findbugs
Signed-off-by: Daan Hoogland <daan.hoogland@gmail.com>

This closes #539
2015-07-09 15:01:58 +02:00
Daan Hoogland
d8ab5f3ab8 findbugs: equals unit tests
Signed-off-by: Daan Hoogland <daan@onecht.net>
2015-07-07 13:48:19 +02:00
Daan Hoogland
34629446b6 findbugs equals(obj) implementations go against the contract these are removed from the call into separate calls
Signed-off-by: Daan Hoogland <daan@onecht.net>
2015-07-07 13:48:19 +02:00
Likitha Shetty
13a98dd196 CLOUDSTACK-8601. VMFS storage added as local storage can be re-added as shared storage.
Fail addition of a VMFS shared storage pool in case it has already been added as local storage in CS.
2015-07-01 10:47:36 +05:30
Rohit Yadav
107595a6a5 CLOUDSTACK-8457: SAML auth plugin improvements for production usage
* Move config options to SAML plugin
  This moves all configuration options from Config.java to SAML auth manager. This
  allows us to use the config framework.
* Make SAML2UserAuthenticator validate SAML token in httprequest
* Make logout API use ConfigKeys defined in saml auth manager
* Before doing SAML auth, cleanup local states and cookies
* Fix configurations in 4.5.1 to 4.5.2 upgrade path
* Fail if idp has no sso URL defined
* Add a default set of SAML SP cert for testing purposes
  Now to enable and use saml, one needs to do a deploydb-saml after doing a deploydb
* UI remembers login selections, IDP server

- CLOUDSTACK-8458:
    * On UI show dropdown list of discovered IdPs
    * Support SAML Federation, where there may be more than one IdP
        - New datastructure to hold metadata of SP or IdP
        - Recursive processing of IdP metadata
        - Fix login/logout APIs to get new interface and metadata data structure
        - Add org/contact information to metadata
        - Add new API: listIdps that returns list of all discovered IdPs
        - Refactor and cleanup code and tests

- CLOUDSTACK-8459:
    * Add HTTP-POST binding to SP metadata
    * Authn requests must use either HTTP POST/Artifact binding

- CLOUDSTACK-8461:
    * Use unspecified x509 cert as a fallback encryption/signing key
      In case a IDP's metadata does not clearly say if their certificates need to be
      used as signing or encryption and we don't find that, fallback to use the
      unspecified key itself.

- CLOUDSTACK-8462:
    * SAML Auth plugin should not do authorization
      This removes logic to create user if they don't exist. This strictly now
      assumes that users have been already created/imported/authorized by admins.
      As per SAML v2.0 spec section 4.1.2, the SP provider should create authn requests using
      either HTTP POST or HTTP Artifact binding to transfer the message through a
      user agent (browser in our case). The use of HTTP Redirect was one of the reasons
      why this plugin failed to work for some IdP servers that enforce this.
    * Add new User Source
      By reusing the source field, we can find if a user has been SAML enabled or not.
      The limitation is that, once say a user is imported by LDAP and then SAML
      enabled - they won't be able to use LDAP for authentication
    * UI should allow users to pass in domain they want to log into, though it is
      optional and needed only when a user has accounts across domains with same
      username and authorized IDP server
    * SAML users need to be authorized before they can authenticate
        - New column entity to track saml entity id for a user
        - Reusing source column to check if user is saml enabled or not
        - Add new source types, saml2 and saml2disabled
        - New table saml_token to solve the issue of multiple users across domains and
          to enforce security by tracking authn token and checking the samlresponse for
          the tokens
        - Implement API: authorizeSamlSso to enable/disable saml authentication for a
          user
        - Stubs to implement saml token flushing/expiry

- CLOUDSTACK-8463:
    * Use username attribute specified in global setting
      Use username attribute defined by admin from a global setting
      In case of encrypted assertion/attributes:
      - Decrypt them
      - Check signature if provided to check authenticity of message using IdP's
        public key and SP's private key
      - Loop through attributes to find the username

- CLOUDSTACK-8538:
    * Add new global config for SAML request sig algorithm

- CLOUDSTACK-8539:
    * Add metadata refresh timer task and token expiring
        - Fix domain path and save it to saml_tokens
        - Expire hour old saml tokens
        - Refresh metadata based on timer task
        - Fix unit tests

This closes #489

(cherry picked from commit 20ce346f3a)
Signed-off-by: Rohit Yadav <rohit.yadav@shapeblue.com>

Conflicts:
	client/WEB-INF/classes/resources/messages_hu.properties
	plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resource/wrapper/xenbase/CitrixCheckHealthCommandWrapper.java
	plugins/user-authenticators/saml2/src/org/apache/cloudstack/api/command/SAML2LoginAPIAuthenticatorCmd.java
	ui/scripts/ui-custom/login.js
2015-06-29 12:31:51 +02:00
Daan Hoogland
f5847147c0 findbugs: serializable fields
Signed-off-by: Daan Hoogland <daan@onecht.net>

This closes #533
2015-06-26 15:11:16 +02:00
wilderrodrigues
cbf5155f7b Renaming a couple of varibale to avoid misunderstanding
- All "micro" references have been replaced with "nano"

Signed-off-by: Daan Hoogland <daan@onecht.net>

This closes #516
2015-06-24 09:36:41 +02:00
wilderrodrigues
5557ad5588 Offer a getDurationInMillis() method in the Profiler utility class
- New implementation uses nanoseconds. Due to that, the places where the Profiler is used as a Monitor and/or
     a stopwatch will suffer with the difference in the return
   - Also added a getDuration(), which returns the time in nanoseconds in case someone wants to use it instead
   - Added an extra test to check if the getDuration() works fine with nanoseconds
   - Fixed the test that checks the time in milliseconds: I added an error margin to cover the test better

Signed-off-by: Daan Hoogland <daan@onecht.net>
2015-06-24 09:36:40 +02:00
Daan Hoogland
e3daa10960 findbugs: final fields
Signed-off-by: Daan Hoogland <daan@onecht.net>

This closes #517
2015-06-24 09:29:56 +02:00