Using Stunnel with Perforce

I need to set up bi-directional communications between a Perforce client and a Perforce server across an untrusted network. How do I secure the Perforce network transport?

In order to secure communication between a Perforce client and Server (p4d), you must use a third-party tool to encrypt network traffic between the two. Network encryption tools include ssh, proprietary VPNs, and Stunnel. This article describes the latter.

Note that the following applies to using stunnel to secure a Perforce connection.  For securing P4Web (i.e., setting it up to use https, see: Creating and Optimizing a Secure P4Web Connection With "stunnel".

Stunnel is an open-source encryption package that allows users to set up SSL tunnels between client(s) and server(s). Using stunnel allows you to set up a port that accepts SSL connections from an SSL-enabled client or another stunnel server. Because both the Perforce server (p4d) and its clients (p4, p4v, p4win) do not support SSL, this article demonstrates how to set up two stunnel servers to talk to each other:

One on the client machine, to accept client requests, encrypt them, and forward them on.One on the server machine, which accepts the encrypted connection, decrypts it and passes it on to the Perforce server.The following information is used to demonstrate how to set up Stunnel: The server machine in named foo. It runs Perforce on localhost:1666, and wants to accept incoming SSL connections for Perforce on foo:2666.The client machine is named bar. Stunnel will be set up so that client requests to localhost:1666 are forwarded, encrypted, to foo:2666 (the server machine).
Why set up Perforce on localhost:1666? This prevents anyone from contacting the server without first going through the stunnel. However, this is not a requirement - stunnel can forward the connection to any host and port.

Before you can set up stunnel, you need to create a self-signed SSL certificate for the stunnel server to provide to stunnel clients contacting it. While any SSL software should be able to do this, the quickest way to do it is via the OpenSSL package. When you have OpenSSL installed, run the following command to generate your certificate:

openssl req -new -x509 -days 365 -nodes -out stunnel.pem -keyout stunnel.pem

When you have the certificate, you are ready to set up Stunnel.

The most recent version of Stunnel, as of this writing, is 4.34. The following information applies to version 4.x of Stunnel. Version 3.x uses a substantially different format, please see the Version 3 section below for information on how to set up version 3 of stunnel.

Stunnel is available as part of the base distribution for a lot of Linuxes and some Unixes.  Check the documentation for your particular OS, or go and grab the source.

Windows users should use this package: http://www.stunnel.org/download/binaries.html

Place the following in a file named "stunnel_client.cnf". Place that file somewhere that stunnel can access it.

; stunnel_client.cnfpid=/var/run/stunnel.pid[p4]accept=localhost:1666connect=foo:2666client=yesStart the client-side stunnel on Linux/Unix with: stunnel /stunnel_client.cnf

Client Configuration (Windows)

cert = stunnel.pemsocket = l:TCP_NODELAY=1socket = r:TCP_NODELAY=1debug = 7output = stunnel.logclient = yes[p4s]accept = 1666connect = :2666

Now start the stunnel program.

Any client requests to port 1666 on the local machine are encrypted and forwarded to foo:2666.

Place the following in a file named "stunnel_server.cnf", and place that file somewhere that stunnel can access it.

; stunnel_server.cnfpid=/var/run/stunnel.pid[p4d]cert=/etc/ssl/certs/stunnel.pemaccept=2666connect=localhost:1666

Start the server the same way as the client:

stunnel /stunnel_server.cnf

Now the server is set up to listen for incoming SSL requests to port 2666 and forward them on to port 1666 on the localhost.

Edit the service's "stunnel.conf" and place the following within it: cert = stunnel.pemkey = stunnel.pemsocket = l:TCP_NODELAY=1socket = r:TCP_NODELAY=1debug = 7output = stunnel.logclient = no[p4s]accept = 2666 connect = 1666 

Now start the stunnel program.  Incoming requests to port 2666 will now be decrypted and sent to port 1666.

If you are using version 3 of Stunnel, then you do not need to set up configuration files - you can specify everything on the command line.

stunnel -p /stunnel.pem -d 2666 -r localhost:1666

The above command sets up stunnel to listen to port 2666 and pass connections on to localhost:1666.

stunnel -c -d localhost:1666 -r foo:2666

The above command sets up stunnel to forward requests to port 1666 on to foo:2666.

View the original article here

Using the server flag -Cn

Please note that the use of -C1 is documented, however the use of other -Cn options are still undocumented, and are subject to change or removal without prior notice. Consider using the p4migrate utility instead to translate case. If you choose to use the -Cn flag, you must ALWAYS use this flag.  This is necessary both when starting your Perforce Server and when running checkpointing or upgrade operations.  If you do not do so, you risk corrupting your Perforce database and/or your checkpoint and journal.

What does the -Cn flag do?

When do I use it?

The -Cn flag instructs the Perforce Server to override the operating system's case handling behavior for the Perforce database. The flag was introduced in release 2004.2, and is a server side flag. From the release notes:

Forces the server to operate in case-sensitive (n=0) on Windows orcase-insensitive (n=1) mode on Unix. -C2 is an experimental 'hybrid'order: case folding but uniqueness preserving.

You use the -Cn flag when:

You want to change the way the Perforce Server handles the case, overriding the operating system's default case-handling.
You are migrating your Perforce Server to an operating system with a different case-handling method, but you wish to retain the existing method for case-handling.

You do NOT use the -Cn flag when:

You are not intending to override the operating system case-handling.

If you do not intend to override case-handling, you checkpoint the database, resolve any case conflicts and inconsistencies, then restore from this case-consistent checkpoint. This process is explained in more detail in Article #75: Cross-Platform Server Migration.

You MUST NOT use -C0 or -C2 when:

running your Perforce Server on a Windows platform, and one file name or path differs from another only in case. This will corrupt the corresponding archive files.

When using either -C0 or -C2, the Perforce Server makes a distinction between "//depot/file.txt" and "//depot/FILE.TXT", and tells the operating system to write "file.txt,v" or "FILE.TXT,v" accordingly. Although Windows is capable of preserving the case used, it is unaware of the difference between these two names. Therefore, "file.txt,v" overwrites or is appended to "FILE.TXT,v", and so too for the reverse, causing loss or corruption of revision content.

If you have not created a database, you can simply start the Perforce Server using the appropriate -Cn flag.

For an existing database, call or email Perforce Technical Support at support@perforce.com because this is not a trivial task. You will need to take a checkpoint of the database, then restore it including the appropriate -Cn flag in the "restore" command. This restore ensures that the order of the keys is appropriate for the case-handling. In addition, you will need to correct case differences using the p4migrate script, and change versioned file line endings.

Important: The -Cn flag can only be used against new Perforce databases. Using the -Cn flag against a Perforce Server that is already in use instantly renders the keys out of order. Not only does it make the data largely inaccessible, it could easily cause the btree code enough confusion to corrupt the tree.

Notes:

The 2007.2 release of the Perforce Server checks the case order of the database on startup. If there is a mismatch, an error is reported in the log:

Perforce server error:Database open error on db.counters!BTree Case Order Mismatch
As of 2010.1 -C1 is now documented in p4d -h $p4d -h$ -C1 Force server to be case insensitive

For a working example of using the -Cn flag see Article 516: Moving from Windows to Linux, Retaining Case-handling.

3 users have rated this article 3.3 out of 5

View the original article here

Using P4Broker With Replica Servers

Article #:1354Created:12/08/10Modified:12/08/10

One of the most common uses of using a Perforce replica server is to re-direct read-only commands using the P4Broker application. This allows Perforce administrators to create a single server and host address for their users, and transparently enforce key commands to use the appropriate Perforce server to reduce overall server load.

For this example:

The broker is at port "broker:33333"The production server is at port "master:11111"The read only replica is at port "replica:22222"

Installing and configuring P4Broker:

Install the P4Broker software for your platform. Create a blank broker configuration file: p4broker -C > /p4broker/root/p4broker.confEdit the P4Broker configuration file to add the target, listen port, and other broker information: target = master:11111;listen = 33333;directory = /p4broker/root/;logfile = broker.log;debug-level = server=1;admin-name = "your name";admin-phone = x1234;admin-email = your.name@yourcompany.com;redirection = selective;

Add an "altserver" for the replica:

altserver: replica1{ target = replica:22222;}

Add command handlers to re-direct read only commands to the replica:

command: annotate{ action = redirect; destination = replica1;}command: branches{ action = redirect; destination = replica1;}command: changes{ action = redirect; destination = replica1;}command: clients{ action = redirect; destination = replica1;}command: counters{ action = redirect; destination = replica1;}command: depots{ action = redirect; destination = replica1;}command: describe{ action = redirect; destination = replica1;}command: diff2{ action = redirect; destination = replica1;}command: dirs{ action = redirect; destination = replica1;}command: filelog{ action = redirect; destination = replica1;}command: files{ action = redirect; destination = replica1;}command: show{ action = redirect; destination = replica1;}command: fstat{ action = redirect; destination = replica1;}command: grep{ action = redirect; destination = replica1;}command: groups{ action = redirect; destination = replica1;}command: jobs{ action = redirect; destination = replica1;}command: labels{ action = redirect; destination = replica1;}command: print{ action = redirect; destination = replica1;}command: sizes{ action = redirect; destination = replica1;}command: fixes{ action = redirect; destination = replica1;}command: verify{ action = redirect; destination = replica1;}command: where{ action = redirect; destination = replica1;} command: workspaces{ action = redirect; destination = replica1;} command: users{ action = redirect; destination = replica1;}# Read/Write Commands with Read-Only flags:## sync -p does not write anything to db.have, so it goes to the replica:command: sync{ flags = -p; action = redirect; destination = replica1;}# Specifications using the -o flag to output to STDOUT also do not write to the DB:command: *{ flags = -o; action = redirect; destination = replica1;}Start the broker: p4broker -c /p4broker/root/p4broker.conf &

Now any read-only commands directed to the P4Broker at port "broker:33333" are directed to the replica at port "replica:22222".

Note: Any commands not matching the above handlers are re-directed to the master server "master:11111" by default.

Details on how to use P4Broker can be found here.

Details on setting up a replica server can be found in the replication chapter of the Perforce System Administrator's Guide.

View the original article here

error using perforce within emacs

hi - I've obtained p4.el and put the following in my .emacs, running under cygwin:

(setq p4-executable "p4")(load-library "p4")

p4.exe is the cygwin version.

On running M-x p4-info I get the following error:

/path/to/p4.exe: error while loading shared libraries: ?: cannot open shared object file: No such file or directory

Yet running M-! p4 info works fine. Probably I'm missing an env variable. Apologies if this is a basic question - I'm new to emacs.

View the original article here

Determining Which File Was Synced Using Overlay Mapping

Several overlay mappings are being used and updated in the client view, and I need to determine which depot file was actually synced to the local workspace.

When overlay mapping is used in the selected workspace client specification, it can become difficult to determine what depot file was synced to a workspace.

Using p4 fstat against a local path that has been overlaid produces information for both files in the depot.

The -Rh flag with p4 fstat limits the output to files actually synced to the client.

For example, assuming a client view of:

+//depot/a.txt //a_client/foo.txt +//depot/b.txt //a_client/foo.txt 

Syncing this client and entering the command produces:

p4 fstat -T "depotFile" -Rh //a_client/foo.txt... depotFile //depot/b.txt 

To confirm that the synced file is what is displayed, you can swap the views:

+//depot/b.txt //a_client/foo.txt  +//depot/a.txt //a_client/foo.txt 

And force sync the file:

p4 sync -f //a_client/foo.txt

The resulting fstat output is:

p4 fstat -T "depotFile" -Rh //a_client/foo.txt... depotFile //depot/a.txt

For more information on overlay mapping, refer to the P4 User's Guide section "Refining client views".

View the original article here

Lock Source file in Perforce while using Maven release plugin

Hi,

I got an error in Hudson console out saying the build fail because Maven release plugin cannot update POM.xml version while there are still unresolved files. The problem is caused by other people updating the POM during the build.

Can any guru teach me how to solve the problem? I am think about a P4 lock, anyone can give me more details of using P4 lock with Maven and hudson?

View the original article here

Installing P4Eclipse Using an HTTP Proxy

The Eclipse software update manager can fail when installing P4Eclipse from behind a firewall. This failure might indicate that additional configuration is required to enable Eclipse to access the internet.

To enable Eclipse to access the Internet from behind a firewall, you can set some proxy settings to take advantage of your organization's HTTP proxy, if one is available. To enable an HTTP proxy connection, perform the following steps:

Mac Users: Please note that your Preferences window is found under Eclipse | Preferences, rather than the Window menu.

Eclipse 3.5.x and 3.6.x:Choose: Window | Preferences | General | Network Connections
Select Manual in the Active Provider dropdown. Edit the appropriate Schema and specify the proxy host address and port number, and authentication, if needed.
Eclipse 3.3.x and 3.4.x:Choose: Window | Preferences | General | Network Connection
Select Manual proxy configuration and specify the HTTP proxy host address and port number.
Eclipse 3.2.x and previous versions:Choose: Window | Preferences | Install/Update
Check Enable HTTP proxy connection and specify the HTTP proxy host address and port number.

For more information on installing P4Eclipse, please see the P4Eclipse Release Notes.

View the original article here

Workflow for renaming a class when using Visual Studio with Perforce

If you have a new enough version of Perforce, it now supports the move command, which will maintain file history across different names. If this is available, your steps are straightforward and should work just fine (from the Perforce side; I'm less experienced with VS).

If you don't have the move command available, you might want to look at this question for a further discussion on renaming/editing files in Perforce.

View the original article here

Perforce: Avoid using the same absolute local path on all dev machines?

Create a workspace with a root of null and map the depot to that client. Here's an example client spec:

Client: client_nameUpdate: 2010/10/20 14:18:23Access: 2010/10/20 14:20:53Owner: ravenHost: ravens-pcDescription: Created by raven.Root: nullOptions: noallwrite noclobber nocompress unlocked modtime rmdirSubmitOptions: leaveunchangedLineEnd: localView: //depot/... //client_name/...

The first thing your script should do is switch to this client by setting the $P4CLIENT$ environment variable:

p4 set p4client=client_name

You script will now be working in the context of that client. By virtue of the fact that you have specified a root of null, any syncing that you do will be relative to the current directory. So, if you sync //depot/foo/... while in C:\bar, your files will appear in C:\bar\foo.

The last thing your script should do is clear the P4CLIENT variable to restore the system to the default client specification:

p4 set p4client=

View the original article here

Using the P4DTG test tool

The P4DTG test application (p4dtg-test) is a utility used to test the basic operation of the P4DTG system. You will find p4dtg-test located in the home directory of a P4DTG installation. It can be a valuable tool when attempting to debug replication problems, or if you are building your own plug-in using the P4DTG SDK. You can use the test application to directly query and test the various plug-in interfaces.

The interfaces provided by p4dtg-test fall into the following categories:

Library
Exercise the plug-in specific functions. These do not require a connection to any specific server.

Attribute
Functions available for listing, validating, and setting data source attributes prior to connection.

Server
Functions that are available after a base connection to a Defect Tracking Source (DTS) server has been established.

Project
Functions that are available after a connection to a specific project on a connected server has been made.

Defect
Functions that are available after a specific defect from a specific project on a connected server has been retrieved from the DTS server.

For a full list of supported commands, see the online help by starting p4dtg-test then entering help at the prompt:

C:> p4dtg-test.exe> help

Most invocations of p4dtg-test will follow a simple pattern:

Load the plug-in library (LL)

You can either start p4dtg-test with no parameters, and then use LL to load the plug-in:

WINDOWS:

C:> p4dtg-test.exe> LL .\plugins\p4jobdt.dllLoading module: [.\plugins\p4jobdt.dll]Module loaded: .\plugins\p4jobdt.dll>

LINUX/BSD:

$ p4dtg-test> LL ./plugins/p4jobdt.soLoading module: [./plugins/p4jobdt.so]Module loaded: ./plugins/p4jobdt.so>or pass the plug-in you want to load when starting p4dtg-test: C:> p4dtg-test.exe .\plugins\p4jobdt.dllLoading module: [.\plugins\p4jobdt.dll]Module loaded: .\plugins\p4jobdt.dll>$ p4dtg-test ./plugins/p4jobdt.soLoading module: [./plugins/p4jobdt.so]Module loaded: ./plugins/p4jobdt.so>
 Connect to a data source (LC) > LC server.com:1666 dtguser dtguserConnecting: [server.com:1666:dtguser:dtguser]Connected.
 List available projects, and then load a project (SL, SP) > SLProjects: [Jobs]> SP JobsLoading project: [Jobs]Project loaded.For a Perforce data source, the only project is Jobs. For Bugzilla, the only project is Bugs. For Quality Center and possibly other data sources, you will get a list of defined projects in that data source.
Perform Project and Defect Functions

Some possibilities include:

List the project fields (PS). When connected to a Perforce data source, this is the contents of the job spec. > PSFields: [Job]:[word][4] [Status]:[select][0] [UNCONFIRMED] [NEW] [ASSIGNED] [REOPENED] . . .>
Get a list of defects that have changed since some date/time (PL) > PL 10 2010/07/22/00/00/00 ModDate ModBy dtguserDefects: [job000074] [job000075]
Load a defect and get its contents (PD, DL) > PD job000075Defect loaded:[job000075]> DLDefectFields: [Job]:[job000075] [Status]:[UNCONFIRMED] [Assigned_To]:[bruno] [Date]:[2009/07/13 14:02:57] [Summary]:[Summarize the prize] [Resolution]:[NONE] [Priority]:[P5] [Severity]:[normal] [ModDate]:[2009/07/13 14:02:57] [ModBy]:[bruno] [DTGConfig-User]:[dtguser] [DTGConfig-Server]:[10.0.10.100:1666] [DTGAttribute-Unicode server]:[n] [DTGAttribute-Wait time]:[10] [DTGConfig-Project]:[jobs] [DTGConfig-Project]:[Jobs]
Update a defect (DW) > DW Status ASSIGNEDdefect_set_field(Status):[ASSIGNED]> DLDefectFields: [Job]:[job000075] [ModBy]:[bruno] [Assigned_To]:[bruno] [Date]:[2009/07/22 16:10:13] [Summary]:[Summarize the prize] [Resolution]:[NONE] [Priority]:[P5] [Severity]:[normal] [ModDate]:[2009/07/22 16:10:13] [Status]:[ASSIGNED] [DTGConfig-User]:[dtguser] [DTGConfig-Server]:[10.0.10.100:1666] [DTGAttribute-Unicode server]:[n] [DTGAttribute-Wait time]:[10] [DTGConfig-Project]:[jobs] [DTGConfig-Project]:[Jobs]Clean up and exit

This is dependent on exactly what commands you have run up to this point. For example:

> DFDefect freed.> PFProject freed.> SFDisconnected from server.> LFModule freed> Q

Commands can be added to a text file and passed to p4dtg-test, with output redirected to a file. For example, a file called "commands.txt" with the following contents:

LL .\plugins\p4jobdt.dllLC server.perforce.com:1666 dtguser dtguserSP JobsPSPL 10 2009/07/22/00/00/00 ModDate ModBy dtguserPD job000075DLDFPFSFLFQ

can be passed to p4dtg-test as follows, producing output to "output.txt":

p4dtg-test < commands.txt > output.txt
Unicode-enabled servers
When P4DTG is run against a Unicode-enabled server, you may get a message similar to the below. Error(1): [UsageError]Unicode server permits only unicode enabled clients.This indicates that the plug-in needs to be in Unicode mode and "AS unicode y" needs to be added in the beginning of the test script.

For example: LL .\plugins\p4jobdt.dllAS unicode yLC server.perforce.com:1666 dtguser dtguser (connect to your server)SP Jobs (load projects)PSPL 10 2009/07/22/00/00/00 ModDate ModBy dtguser (list of defects that have changes since some date/time)PD job000075  (load some defect and get its contents)DLDFPFSFLFQ

View the original article here