Perforce Software Build Engineer - Bracknell

Perforce Software Build Engineer - Bracknell

Huxley Associates are recruiting for a Software Build Engineer to work for a key client based in Bracknell.

You will join at a fantastic time as they continue in a period of signifant growth due to the demand for their market leading sofware products. The company previously used Subversion as their source-control system but having moved to Perforce they require someone who can bring expertise in that area they currently do not have.

You will be part of a techncially strong software team who currently look after the build process but they want someone to take the burden from them and allow them to focus on the software development process.

To be considered your CV should demonstrate your experience in a Build Engineer role using Perforce, Hudson and Shell Scripting. If you have any knowledge of Java that would be highly beneficial to help understand the software product.

Salary £40,000 - £45,000 + benefits

Apply NOW!!

View the original article here

Perforce Replication

This section of the Knowledge Base augments the Perforce Replication chapter of the Perforce System Administrator's Guide.

SUMMARY: Replication is the duplication of server metadata from one Perforce Server (the master server) to another Perforce Server (the replica server). The basics of replication are covered in ...

View the original article here

Using Stunnel with Perforce

I need to set up bi-directional communications between a Perforce client and a Perforce server across an untrusted network. How do I secure the Perforce network transport?

In order to secure communication between a Perforce client and Server (p4d), you must use a third-party tool to encrypt network traffic between the two. Network encryption tools include ssh, proprietary VPNs, and Stunnel. This article describes the latter.

Note that the following applies to using stunnel to secure a Perforce connection.  For securing P4Web (i.e., setting it up to use https, see: Creating and Optimizing a Secure P4Web Connection With "stunnel".

Stunnel is an open-source encryption package that allows users to set up SSL tunnels between client(s) and server(s). Using stunnel allows you to set up a port that accepts SSL connections from an SSL-enabled client or another stunnel server. Because both the Perforce server (p4d) and its clients (p4, p4v, p4win) do not support SSL, this article demonstrates how to set up two stunnel servers to talk to each other:

One on the client machine, to accept client requests, encrypt them, and forward them on.One on the server machine, which accepts the encrypted connection, decrypts it and passes it on to the Perforce server.The following information is used to demonstrate how to set up Stunnel: The server machine in named foo. It runs Perforce on localhost:1666, and wants to accept incoming SSL connections for Perforce on foo:2666.The client machine is named bar. Stunnel will be set up so that client requests to localhost:1666 are forwarded, encrypted, to foo:2666 (the server machine).
Why set up Perforce on localhost:1666? This prevents anyone from contacting the server without first going through the stunnel. However, this is not a requirement - stunnel can forward the connection to any host and port.

Before you can set up stunnel, you need to create a self-signed SSL certificate for the stunnel server to provide to stunnel clients contacting it. While any SSL software should be able to do this, the quickest way to do it is via the OpenSSL package. When you have OpenSSL installed, run the following command to generate your certificate:

openssl req -new -x509 -days 365 -nodes -out stunnel.pem -keyout stunnel.pem

When you have the certificate, you are ready to set up Stunnel.

The most recent version of Stunnel, as of this writing, is 4.34. The following information applies to version 4.x of Stunnel. Version 3.x uses a substantially different format, please see the Version 3 section below for information on how to set up version 3 of stunnel.

Stunnel is available as part of the base distribution for a lot of Linuxes and some Unixes.  Check the documentation for your particular OS, or go and grab the source.

Windows users should use this package: http://www.stunnel.org/download/binaries.html

Place the following in a file named "stunnel_client.cnf". Place that file somewhere that stunnel can access it.

; stunnel_client.cnfpid=/var/run/stunnel.pid[p4]accept=localhost:1666connect=foo:2666client=yesStart the client-side stunnel on Linux/Unix with: stunnel /stunnel_client.cnf

Client Configuration (Windows)

cert = stunnel.pemsocket = l:TCP_NODELAY=1socket = r:TCP_NODELAY=1debug = 7output = stunnel.logclient = yes[p4s]accept = 1666connect = :2666

Now start the stunnel program.

Any client requests to port 1666 on the local machine are encrypted and forwarded to foo:2666.

Place the following in a file named "stunnel_server.cnf", and place that file somewhere that stunnel can access it.

; stunnel_server.cnfpid=/var/run/stunnel.pid[p4d]cert=/etc/ssl/certs/stunnel.pemaccept=2666connect=localhost:1666

Start the server the same way as the client:

stunnel /stunnel_server.cnf

Now the server is set up to listen for incoming SSL requests to port 2666 and forward them on to port 1666 on the localhost.

Edit the service's "stunnel.conf" and place the following within it: cert = stunnel.pemkey = stunnel.pemsocket = l:TCP_NODELAY=1socket = r:TCP_NODELAY=1debug = 7output = stunnel.logclient = no[p4s]accept = 2666 connect = 1666 

Now start the stunnel program.  Incoming requests to port 2666 will now be decrypted and sent to port 1666.

If you are using version 3 of Stunnel, then you do not need to set up configuration files - you can specify everything on the command line.

stunnel -p /stunnel.pem -d 2666 -r localhost:1666

The above command sets up stunnel to listen to port 2666 and pass connections on to localhost:1666.

stunnel -c -d localhost:1666 -r foo:2666

The above command sets up stunnel to forward requests to port 1666 on to foo:2666.

View the original article here

Perforce Metadata Replication

Replication is the duplication of server metadata from one Perforce Server (the master server) to another Perforce Server (the replica server). The basics of replication are covered in Chapter 10 of the Perforce System Administrator's Guide. This article provides answers to common questions that come up when setting up and implementing Perforce metadata replication.

With the release of Perforce Server 2009.2, the p4 replicate utility is the supported method of Perforce metadata replication. Information on system requirements and limits and restrictions when using p4 replicate can be found in the Perforce System Administrator's Guide.

The functionality provided by p4 replicate was first introduced in beta (prior to the release of Perforce Server 2009.2) as a standalone utility called p4jtail. p4jtail is now available as open source in the Public Depot as an example of a client program that uses the Perforce C++ API to do replication. It also serves as a starting point for those interested in extending or customizing replication behavior in some way not supported by p4 replicate.

Prior to p4 replicate and p4jtail, the common utility used for replication was p4jrep, a user-contributed Perforce journal replication utility available from the Public Depot. Both p4jrep and p4jtail remain unsupported but customers can obtain assistance and advice on usage by contacting Perforce Technical Support.

What is p4 replicate?
Where do I run the p4 replicate command?
Does p4 replicate work against archive file content?
Can I replicate from one master server to multiple replica servers?
Can I replicate bidirectionally?
Can I checkpoint the master server?
Can I checkpoint the replica server?
How does p4 replicate extract journal records from the master server?
How does p4 replicate locate journal files on the master server?
How does p4 replicate track what journal records have been processed?
Can I filter what gets replicated from the master server?
What is the upgrade process when doing replication?

Q. What is p4 replicate?

A. p4 replicate is a client command that polls a primary Perforce Server (the master) for new journal records, then provides these records to a subprocess that restores the records into the database of a Secondary Perforce Server (the replica).

Q Where do I run the p4 replicate command?

A. You run p4 replicate on the machine that will host the replica Perforce Server. Because p4 replicate is a client command, you can run it from any client machine that can connect to the master server.

Q. Does p4 replicate work against archive file content?

A. No. p4 replicate only replicates Perforce Server metadata, that is the information contained in the db.* files. Depending on the requirements for your replica server, you can optionally configure the replica server to read the master server's versioned files, or use third-party or native OS functionality to mirror the versioned file content to the replica server.

Q. Can I replicate from one master server to multiple replica servers?

A. Yes, multiple instances of p4 replicate can be run against the same master server.

Q. Can I replicate bidirectionally?

A. No, replication is unidirectional.

Q. Can I checkpoint the master server?

A. Yes, p4 replicate continues to run during journal rotation on the master server, except when p4 replicate is started using the -x option. If p4 replicate is started with the -x option, replication will terminate when journal rotation is detected on the master server.

Can I checkpoint the replica server?

A. Yes. If using the replica server for offline checkpoints, checkpoint using the p4d -jd option. See the Offline Checkpoints Knowledge Base article for further details.

Q. How does p4 replicate extract journal records from the master server?

A. Behind the scenes, p4 replicate makes use of the p4 export command to extract journal records from the master server. In addition to providing journal records, p4 export adds fields to its output to indicate points of transactional consistency and the end of the journal file.

Q. How does p4 replicate locate journal files on the master server?

A. p4 replicate locates the live journal file on the master server based on the setting of P4JOURNAL or the -J option passed to p4d on startup. For rotated journals, p4 replicate looks by default in the P4ROOT directory on the master server and uses the default naming convention for rotated journals (journal.nnn). If using the prefix option when checkpointing or rotating the journal on the master server (prefix in this example is /data/backup/p4):

p4d -jc /data/backup/p4 ORp4 admin checkpoint /data/backup/p4p4d -jj /data/backup/p4 ORp4 admin journal /data/backup/p4

the name and possibly the location of rotated journals will not follow the default convention. In this case, the -J flag to p4 replicate must be used, specifying the same prefix used in the checkpoint and journal rotation commands on the master server:

p4 replicate -J /data/backup/p4 ...

Q. How does p4 replicate track what journal records have been processed?

A. p4 replicate uses a statefile to store a journal position token consisting of a journal number and position (byte) offset indicating what journal records from the master have been processed.

Q. Can I filter what gets replicated from the master server?

A. Yes. Filters can be used with p4 replicate to customize the journal records that are replicated. The command that p4 replicate runs on the replica to replay journal records could be, for example, a script that first applies some filter to the journal data stream and then passes the data to the command used to replay the journal records into the replica.

Warning: If you are using replicate to perform offline checkpoints, then do not filter the journal stream. By definition, filters remove content; therefore, a filtered journal replication will not provide sufficient data integrity for your checkpoint and backup process. The primary use of journal stream filtering is to support replication to a read-only Perforce Server used for build or reporting purposes. Depending on the specific use case, it might be desirable to filter out db.have or other db table entries.

Q. What is the upgrade process when doing replication?

A.  For detailed upgrade procedures, see the Upgrading replica servers section of the System Administrator's Guide.

View the original article here

Installer problems on Vista

Perforce Installer versions 2005.1 and earlier do not run by default on Windows Vista because they do not recognize the OS version. Running these older installers will generate the following error message:

Perforce is only supported on Windows NT/2000/XP/2003 and Windows 98/ME.Windows 95 and Windows 98 Retail Edition are not supported.This installation will not continue

To work around this installation problem, right-click on the Installer executable (perforce.exe) in Explorer, select Properties, and go to the "Compatibility" tab. Set the installer to run in XP compatibility mode with full administrator privileges.

An alternative is to simply use a later version of Perforce and the Perforce installer. As of the 2005.2 version of Perforce, there is no problem running the Perforce Installer on Windows Vista.

View the original article here

Using the server flag -Cn

Please note that the use of -C1 is documented, however the use of other -Cn options are still undocumented, and are subject to change or removal without prior notice. Consider using the p4migrate utility instead to translate case. If you choose to use the -Cn flag, you must ALWAYS use this flag.  This is necessary both when starting your Perforce Server and when running checkpointing or upgrade operations.  If you do not do so, you risk corrupting your Perforce database and/or your checkpoint and journal.

What does the -Cn flag do?

When do I use it?

The -Cn flag instructs the Perforce Server to override the operating system's case handling behavior for the Perforce database. The flag was introduced in release 2004.2, and is a server side flag. From the release notes:

Forces the server to operate in case-sensitive (n=0) on Windows orcase-insensitive (n=1) mode on Unix. -C2 is an experimental 'hybrid'order: case folding but uniqueness preserving.

You use the -Cn flag when:

You want to change the way the Perforce Server handles the case, overriding the operating system's default case-handling.
You are migrating your Perforce Server to an operating system with a different case-handling method, but you wish to retain the existing method for case-handling.

You do NOT use the -Cn flag when:

You are not intending to override the operating system case-handling.

If you do not intend to override case-handling, you checkpoint the database, resolve any case conflicts and inconsistencies, then restore from this case-consistent checkpoint. This process is explained in more detail in Article #75: Cross-Platform Server Migration.

You MUST NOT use -C0 or -C2 when:

running your Perforce Server on a Windows platform, and one file name or path differs from another only in case. This will corrupt the corresponding archive files.

When using either -C0 or -C2, the Perforce Server makes a distinction between "//depot/file.txt" and "//depot/FILE.TXT", and tells the operating system to write "file.txt,v" or "FILE.TXT,v" accordingly. Although Windows is capable of preserving the case used, it is unaware of the difference between these two names. Therefore, "file.txt,v" overwrites or is appended to "FILE.TXT,v", and so too for the reverse, causing loss or corruption of revision content.

If you have not created a database, you can simply start the Perforce Server using the appropriate -Cn flag.

For an existing database, call or email Perforce Technical Support at support@perforce.com because this is not a trivial task. You will need to take a checkpoint of the database, then restore it including the appropriate -Cn flag in the "restore" command. This restore ensures that the order of the keys is appropriate for the case-handling. In addition, you will need to correct case differences using the p4migrate script, and change versioned file line endings.

Important: The -Cn flag can only be used against new Perforce databases. Using the -Cn flag against a Perforce Server that is already in use instantly renders the keys out of order. Not only does it make the data largely inaccessible, it could easily cause the btree code enough confusion to corrupt the tree.

Notes:

The 2007.2 release of the Perforce Server checks the case order of the database on startup. If there is a mismatch, an error is reported in the log:

Perforce server error:Database open error on db.counters!BTree Case Order Mismatch
As of 2010.1 -C1 is now documented in p4d -h $p4d -h$ -C1 Force server to be case insensitive

For a working example of using the -Cn flag see Article 516: Moving from Windows to Linux, Retaining Case-handling.

3 users have rated this article 3.3 out of 5

View the original article here

How to Monitor a Swamped Perforce Server

How to monitor a swamped Perforce server not responding to p4 monitor show commands. In most circumstances, you can run the p4 monitor command to identify the client processes currently running on your Perforce Server. However, it is possible that a long-running Perforce command can block other Perforce processes, including the p4 monitor request.

It is possible to prevent the blocking of p4 monitor on a Perforce Server running on UNIX or MacOS (or other operating systems that natively support symbolic links). To do so, you must create a new Perforce Server instance that shares the db.monitor table by means of a symlink.

Below is an example of configuring a new "monitor server". Note the new monitor server runs in a different P4ROOT (/perforce/monitor) location and on a different port (1999) than the production server. # Create a new monitor server P4ROOT directory #mkdir /perforce/monitor# Link the production server monitor table into the new P4ROOT directory#ln -s /perforce/production/db.monitor /perforce/monitor/db.monitor# Start up the new "monitor server" (with journaling off)#p4d -r /perforce/monitor -J off -p 1999 -d# Remember to enable monitoring by setting the monitor counter to 1 on the new server#p4 -p 1999 counter -f monitor 1# Stop and restart the monitor server for the new monitor setting to take effect#p4 -p 1999 admin stopp4d -r /perforce/monitor -J off -p 1999 -dOnce the monitor server instance is set up (as above), you can then monitor your production server by running the p4 monitor command on the new monitor server. For example: p4 -p 1999 monitor show

View the original article here