Nevermind.dk


Tutorial: Moving a Centos physical server with Soft RAID to be a KVM virtual machine guest using virt-p2v and virt-v2v


I have used KVM for several years on Centos as a platform for running virtualized servers, and my experiences have been very good

Now I have the case where I want to move a physical Centos server running RAID 10 on soft RAID, to a Centos KVM server.



There are several ways to create a VM guest from physical server however it is a complex process and plenty of things can go wrong on the way.

The safest way to do it is to use virt-v2v and virt-p2v.

These are Red Hat created tools but can be downloaded even if you are not a Red Hat user.

The (source) physical server will be running virt-p2v (physical to virtual) and the conversion server (in this case also the target server) will do the conversion to be a KVM guest

Source - the physical server:

The source is the physical server.

- To handle to the transition to be a KVM virtual machine guest, you need to shutdown the running server.
- The physical server will boot up on virt-p2v
Read about virt-p2v here: http://libguestfs.org/virt-p2v.1.html
- Download virt-p2v ISO image here: http://oirase.annexia.org/virt-p2v/
- Create a bootable device from the ISO image. I use Rufus to do this (https://rufus.akeo.ie/)
- Have the server up in virt-p2v from the bootable device just created
- You will be sending a LOT of data over the network, so a fast network is good and make sure the server can "see" each other (firewall, network segments..)



1) enter the IP of the conversion server
2) enter user name on the conversion server (root)
3) password of user on conversion server.
4) test that you can connect to conversion server.

When OK press "Next" you will see the next and last setup screen


I will show what to fill in.


1) First give the VM a name
2) This is the number of virtual CPUs
3) Amount of RAM

(You can always change the numbers after the VM has been created)

Output options


1) There are several options here


Since I want to create a VM guest on the conversion server I select "libvirt"
2) I leave this empty since I want the process to create a new VM with the name "New Centos Server"
3) Leave blank
4) The format for the VM. The RAW and QCOW2 format are supported. RAW is fastest, but QCOW2 has many more features like snapshots, sparse ..
I choose QCOW2
5) Choose between "Sparse" and "Preallocated". Using Sparse the disk will expand as needed, using less space to start with. However preallocating all the space for disks will speed up write times dramatically.
So if you are using something write heavy use Preallocated.
I choose "sparse" here in this demo.
Virt-p2v are a bit smart an will investigate the source disks and only send real data on disks, not deleted files, empty spaces etc.


Choose which disk to move to VM. I use soft RAID here so it is important to get all disks moved to new server.


Don't move the boot device over, unselect here


Select the network cards to move over

Since KVM may not have the drivers for the physical devices the conversion process will going in an investigate the installtion and substitute with KVM available drivers which means in most
cases Virtio drivers since they will be the fastest drivers. Virtio drivers are Paravirtualized drivers, which gives near "bear metal" performance.
Linux will have the virtio drivers installed already, but if guest is MS Windows you need to download the virtio drivers and install them. You will also need to install libguestfs-winsupport.

Target - the KVM guest

To handle the conversion on the target Centos server you need to install virt-v2v.
Prerequsits: KVM/QEMU is already installed on target server

Installation:
- yum install virt-v2v

To administer the VM guest install virt-manager
- yum install virt-manager

Running the conversion

Go back to the physical server and click on the "Start conversion" button
virt-p2p will report as it goes a long. I will start with the conversion and then moved the disk over.


Any other status than 0 means there was an issue

Open the Virt Manager and voila ! :-)

The server is now running on the target KVM server


Devices on the KVM guest


The is example has been an Centos physical server, but other OS is supported too:

Red Hat Enterprise Linux 3.9
Red Hat Enterprise Linux 4
Red Hat Enterprise Linux 5
Red Hat Enterprise Linux 6
Red Hat Enterprise Linux 7.1 and later
Windows XP
Windows Vista
Windows 7
Windows Server 2003
Windows Server 2008
Windows Server 2008 R2
Windows Server 2012
Windows Server 2012 R2

virt-v2v can of course also move VM guests from other VM platforms:

Red Hat Enterprise Linux 5 Xen
VMware vSphere ESX / ESX(i) - versions 3.5, 4.0, 4.1, 5.0, 5.1, 5.5, 6.0
Published by: Jesper B. Kir at 05/11/2017 12.20.00 Full Post


IBM Domino and DIIOP - not quit doing what it is supposed to.


I am using DIIOP to have remote session with a Domino server, because using Java agent just gave to many issues with some external libraries.


While DIIOP may not be the fastest way to work with data on a IBM Domino server, it usually does the job...well sort of..

I've just found out when using richtextitem.embedobject the name parameter of the file does not work, so the attachment gets this reeeeaaally long name ...including the file path.

Also I needed to create a text file on the server so why not use the Stream class? Unfortunately I got a lot of errors until I found out the Stream was actually not trying to write the file on the Domino server, but my local PC??

If I wanted to write to my local file system I would probably not use a REMOTE DIIOP session! ..capisce IBM?
Published by: Jesper B. Kir at 01/10/2017 23.15.56 Full Post


Maven - how to get dependency JAR files in build too


Maven may be smart to some ....but it is also a bloody XML nightmare, where many things can go wrong ...(or maybe it just me being stupid)

Just a reminder to myself....when I want to have dependency JAR files in build too , add this to pom.xml file



<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<id>copy-dependencies</id>
<phase>prepare-package</phase>
<goals>
<goal>copy-dependencies</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}/classes/lib</outputDirectory>
<overWriteReleases>false</overWriteReleases>
<overWriteSnapshots>false</overWriteSnapshots>
<overWriteIfNewer>true</overWriteIfNewer>
</configuration>
</execution>
</executions>
</plugin>

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-jar-plugin</artifactId>
<configuration>
<archive>
<manifest>
<addClasspath>true</addClasspath>
<!-- <classpathPrefix>lib</classpathPrefix> -->
<!-- <mainClass>test.org.Cliente</mainClass> -->
</manifest>
<manifestEntries>
<Class-Path>lib/</Class-Path>
</manifestEntries>
</archive>
</configuration>
</plugin>
</plugins>
</build>
Published by: Jesper B. Kir at 15/09/2017 13.22.19 Full Post


IBM Domino, a very annoying performance issue now SOLVED


For the past years a customer of mine has had IBM Domino performance issues with a Domino Server.

The issue concerned was really felt when working with attachments.


The company has small offices around the world, so we use SmartUpgrade (which in general works well) to manage Notes Feature Pack upgrades.
We attach the Feature Pack as an attachment in the SmartUpgrade database and use a Policy to push it to users.
Normally this is very fast, but for one server it would maybe take 5 hours to download the file.
The download would start at a decent speed and then only get slower and slower and in the end literally only move a few bytes at the time.
We tried "everything" ...even moving to newer faster hardware did not make any change.

The server is a Windows 2012 R2 server with Domino 9.0.1. FP8 in a Domino cluster. The other server is Linux server which did not have the issue.

The solution
The relevant difference between the two servers Domino in the Notes.ini configuration, was the setting for TCP.
The customer use encryption and compression on the Notes TCP connection

Notes.ini
The Windows server had: TCPIP=TCP,0,15,0,,45088
The Linux server had: TCPIP=TCP, 0, 15, 0,,32800

If looked at the documentation for TCP setting it says:

The TCPIP port line can contain up to six arguments as described below, with the first position numbered as position 0.

Initialization arguments:
argv[0] Driver name
argv[1] Adapter number (unused)
argv[2] Requested number of sessions (unused)
argv[3] Data buffer size to use. If the value is 0, the default size is used. Default sizes are different for different port drivers, as follows:
argv[4] Number of network buffers to preallocate (unused)
argv[5] Port flags, as follows:
0 X 8000 Encryption is enabled
0 X 0020 Compression is requested

Since we use Compression and Encryption of the connection we should have 8020 in hexadecimal for last parameter, which in decimal is 32800.
That is value we had on the fast Linux server. After changing the value on the Windows server to

TCPIP=TCP, 0, 15, 0,,32800

the network was much faster and SmartUpgrade became just as fast as on the Linux server :-)

The big question is why does the Domino installer suggest TCPIP=TCP,0,15,0,,45088 ??
and what are the undocumented Port flags used in this scenario?
Published by: Jesper B. Kir at 29/08/2017 09.52.43 Full Post


IBM Domino lessons learned from using the new FP8 feature of having the view index outside the NSF database (NDX)


At a customer today I wanted to implemented the new IBM Domino feature of having the view index outside the NSF.

First I upgraded to FP8 IF3, then I updated notes.ini from the configuration document with the new setting NIFNSFEnable=1 and restarted the server.


I then wanted to start carefully with one database only ...with something like "load compact -c -nifnsf on databasename.nsf".

Lesson 1)
Unfortunately I misspelled the database path and hit enter. To my surprise the Domino server did not just report back in the console that the database could not be found, ...instead is started compacting on ALL DATABASES!
Not quit what I expected or wanted .... but I let it continue
Lesson learned...software does not always act as expected ...act accordingly..get you spelling in order!

Lesson 2)
I kept an eye on the compacting and I noticed that one big database (GB) just barely got touched only creating a small NDX file and the compacting moved on.
I am a little unsure what to expect now. My guess that it created some kind of "stub", because the database most likely was in use it could not compact it.
So I hope that when DBMT starts compacting it will finish the job....but I am not sure, may need to run the ""load compact -c -nifnsf " again.
Lesson learned ... it is always "fun" when software stops in the middle of doing something ...what is going to happen next?

Lesson 3)
OK this one i primarily my fault,... I got the great idea ..why not open the NDX file in the Domino Designer and have a look around?
Lesson learned... don't do that... your server will come to a grinding halt...
You are probably not meant poke around in a NDX database ;-)

I am sure there is plenty more to be learned...
Published by: Jesper B. Kir at 30/05/2017 15.22.00 Full Post


Making "X-Forwarded-For" log feature IBM Domino FP8 actually work


In March I wrote about the new log feature in IBM Domino in connection with using a Reverse Proxy.

http://nevermind.dk/nevermind/blog.nsf/subject/new-logging-feature-in-ibm-domino-fp8-when-using-reverse-proxy---x-forwarded-for


The issue is that you will always get the IP address of the Reverse Proxy in you HTTP log files and not the actual IP of the user.
This is what the HTTP header "X-Forwarded-For" is used for, however pre FP8 is was not supported on IBM Domino server.
In FP8 it is supported...but not documented.
Thanks to Vitor Pereira who (commented on my blog entry) created a PMR, we now have an answer how to get it working.
Add HTTP_LOG_ACCESS_XFORWARDED_FOR=1 to the servers notes.ini file, restart server and it works!
However if you using domlog.nsf for log data (which is fine for low traffic) you need to change the design to see the X-forwarded-for IP.
You would presume the item would be named something like "XForwardedFor", but no .. it is "ForwardedFor" ..or rather because it is misspelled it is "ForwaredFor" ;-)
Published by: Jesper B. Kir at 17/05/2017 13.04.00 Full Post

Read More