1*0Sstevel@tonic-gate# 2*0Sstevel@tonic-gate# CDDL HEADER START 3*0Sstevel@tonic-gate# 4*0Sstevel@tonic-gate# The contents of this file are subject to the terms of the 5*0Sstevel@tonic-gate# Common Development and Distribution License, Version 1.0 only 6*0Sstevel@tonic-gate# (the "License"). You may not use this file except in compliance 7*0Sstevel@tonic-gate# with the License. 8*0Sstevel@tonic-gate# 9*0Sstevel@tonic-gate# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE 10*0Sstevel@tonic-gate# or http://www.opensolaris.org/os/licensing. 11*0Sstevel@tonic-gate# See the License for the specific language governing permissions 12*0Sstevel@tonic-gate# and limitations under the License. 13*0Sstevel@tonic-gate# 14*0Sstevel@tonic-gate# When distributing Covered Code, include this CDDL HEADER in each 15*0Sstevel@tonic-gate# file and include the License file at usr/src/OPENSOLARIS.LICENSE. 16*0Sstevel@tonic-gate# If applicable, add the following below this CDDL HEADER, with the 17*0Sstevel@tonic-gate# fields enclosed by brackets "[]" replaced with your own identifying 18*0Sstevel@tonic-gate# information: Portions Copyright [yyyy] [name of copyright owner] 19*0Sstevel@tonic-gate# 20*0Sstevel@tonic-gate# CDDL HEADER END 21*0Sstevel@tonic-gate# 22*0Sstevel@tonic-gateDHCP Service Library Synchronization 23*0Sstevel@tonic-gatePeter Memishian, Solaris Software, meem@east.sun.com 24*0Sstevel@tonic-gate 25*0Sstevel@tonic-gate#ident "%Z%%M% %I% %E% SMI" 26*0Sstevel@tonic-gate 27*0Sstevel@tonic-gateIntroduction 28*0Sstevel@tonic-gate============ 29*0Sstevel@tonic-gate 30*0Sstevel@tonic-gateWhen writing DHCP service libraries (i.e., public modules) that provide 31*0Sstevel@tonic-gateaccess to locally-backed datastores (i.e., have their backing datastore on 32*0Sstevel@tonic-gatethe same machine that the module is running on), it can be difficult for 33*0Sstevel@tonic-gatethe module author to synchronize access to the underlying datastore between 34*0Sstevel@tonic-gatemultiple processes, multiple threads within a single process, multiple 35*0Sstevel@tonic-gatethreads within multiple processes, and multiple threads within multiple 36*0Sstevel@tonic-gateprocesses on multiple machines. 37*0Sstevel@tonic-gate 38*0Sstevel@tonic-gateThe goal of DHCP Service Library Synchronization is to simplify the design 39*0Sstevel@tonic-gateof modules using locally-backed datastores by pushing these issues up into 40*0Sstevel@tonic-gatethe DHCP service library framework: by designing your module to use this 41*0Sstevel@tonic-gateframework, your code becomes simpler and your design cleaner. 42*0Sstevel@tonic-gate 43*0Sstevel@tonic-gateWhat does DHCP Service Library Synchronization do for me? 44*0Sstevel@tonic-gate========================================================= 45*0Sstevel@tonic-gate 46*0Sstevel@tonic-gateIt synchronizes access to several of the DHCP Service Library public-layer 47*0Sstevel@tonic-gatefunctions; the particular synchronization guarantees vary depending on the 48*0Sstevel@tonic-gateunderlying function being called: 49*0Sstevel@tonic-gate 50*0Sstevel@tonic-gate add_d?() per-container exclusive-perimeter 51*0Sstevel@tonic-gate delete_d?() per-container exclusive-perimeter 52*0Sstevel@tonic-gate modify_d?() per-container exclusive-perimeter 53*0Sstevel@tonic-gate lookup_d?() per-container shared-perimeter 54*0Sstevel@tonic-gate all others no synchronization provided 55*0Sstevel@tonic-gate 56*0Sstevel@tonic-gateThe term `per-container exclusive perimeter' access means that only one 57*0Sstevel@tonic-gatethread may be inside the per-container "perimeter" at a time; that means 58*0Sstevel@tonic-gatethat if one thread is inside add_dn() for a given container, no other thread 59*0Sstevel@tonic-gatemay be inside add_dn() (or delete_dn(), modify_dn(), and lookup_dn() for 60*0Sstevel@tonic-gatethat same container). However, other threads may be within routines that 61*0Sstevel@tonic-gateprovide no synchronization guarantees such as close_dn(). 62*0Sstevel@tonic-gate 63*0Sstevel@tonic-gateThe term `per-container shared perimeter' access means that multiple threads 64*0Sstevel@tonic-gatemay be inside the perimeter, as long as they are all in routines which have 65*0Sstevel@tonic-gateeither no synchronization guarantees or also have `per-container shared 66*0Sstevel@tonic-gateperimeter' access. For instance, multiple threads may be within lookup_dt() 67*0Sstevel@tonic-gateconcurrently, but another thread may not be in add_dt() at the same time. 68*0Sstevel@tonic-gate 69*0Sstevel@tonic-gateNote that the preceding discussion assumes that all the threads being 70*0Sstevel@tonic-gateserialized are all running on the same machine. However, there's also an 71*0Sstevel@tonic-gateoptional facility which provides synchronization across multiple threads on 72*0Sstevel@tonic-gatemultiple machines as well; see the discussion on cross-host synchronization 73*0Sstevel@tonic-gatebelow. 74*0Sstevel@tonic-gate 75*0Sstevel@tonic-gateHow do I write my module to use DHCP Service Library Synchronization? 76*0Sstevel@tonic-gate===================================================================== 77*0Sstevel@tonic-gate 78*0Sstevel@tonic-gateWrite your module just as you normally would. Of course, when writing your 79*0Sstevel@tonic-gatecode, you get to take advantage of the synchronization guarantees this 80*0Sstevel@tonic-gatearchitecture makes for you. 81*0Sstevel@tonic-gate 82*0Sstevel@tonic-gateWhen you're done writing your module, then add the following to one of your 83*0Sstevel@tonic-gateC source files: 84*0Sstevel@tonic-gate 85*0Sstevel@tonic-gate /* 86*0Sstevel@tonic-gate * This symbol and its value tell the private layer that it must provide 87*0Sstevel@tonic-gate * synchronization guarantees via dsvclockd(1M) before calling our *_dn() 88*0Sstevel@tonic-gate * and *_dt() methods. Please see $SRC/lib/libdhcpsvc/private/README.synch 89*0Sstevel@tonic-gate */ 90*0Sstevel@tonic-gate int dsvc_synchtype = DSVC_SYNCH_DSVCD; 91*0Sstevel@tonic-gate 92*0Sstevel@tonic-gateNext, note that if you want to use cross-host synchronization, you'll need 93*0Sstevel@tonic-gateto bitwise-or in the DSVC_SYNCH_CROSSHOST flag as well -- however, please 94*0Sstevel@tonic-gateread the discussion below regarding cross-host synchronization first! 95*0Sstevel@tonic-gate 96*0Sstevel@tonic-gateThe private layer synchronizes access to similarly named containers; that 97*0Sstevel@tonic-gateis, all requests for a given (location, container_name, container_version, 98*0Sstevel@tonic-gatedatastore) tuple are synchronized with respect to one another. One 99*0Sstevel@tonic-gateimplication of this approach is that there must not be two tuples which 100*0Sstevel@tonic-gateidentify the same container -- for instance, (/var/dhcp, dhcptab, 1, 101*0Sstevel@tonic-gateSUNWfiles) and (/var/dhcp/, dhcptab, 1, SUNWfiles) name the same container 102*0Sstevel@tonic-gatebut are distinct tuples and thus would not be synchronized with respect to 103*0Sstevel@tonic-gateone another! 104*0Sstevel@tonic-gate 105*0Sstevel@tonic-gateTo address this issue, the `location' field given in the above tuple is 106*0Sstevel@tonic-gaterequired to have the property that no two location names map to the same 107*0Sstevel@tonic-gatelocation. Public modules whose `location' field does not meet this 108*0Sstevel@tonic-gateconstraint must implement a mkloctoken() method, prototyped below, which 109*0Sstevel@tonic-gatemaps a location into a token which does meet the constraints. In the above 110*0Sstevel@tonic-gatescenario, mkloctoken() would use realpath(3C) to perform the mapping. 111*0Sstevel@tonic-gate 112*0Sstevel@tonic-gate int mkloctoken(const char *location, char *token, size_t tokensize); 113*0Sstevel@tonic-gate 114*0Sstevel@tonic-gateThe location to map is passed in as `location', which must be mapped into an 115*0Sstevel@tonic-gateASCII `token' of `tokensize' bytes or less. The function should return 116*0Sstevel@tonic-gateDSVC_SUCCESS or a DSVC_* error code describing the problem on failure. Note 117*0Sstevel@tonic-gatethat modules which do not use synchronization or already have location names 118*0Sstevel@tonic-gatewhich meet the constraints need not provide mkloctoken(). 119*0Sstevel@tonic-gate 120*0Sstevel@tonic-gateCross-host Synchronization 121*0Sstevel@tonic-gate========================== 122*0Sstevel@tonic-gate 123*0Sstevel@tonic-gateDatastores wishing to make use of cross-host synchronization have an 124*0Sstevel@tonic-gateadditional constraint: the `location' must be the name of a directory which 125*0Sstevel@tonic-gateis shared and accessible by all hosts which are accessing the datastore. 126*0Sstevel@tonic-gateThis constraint is because the code is uses NFS-based file locking to 127*0Sstevel@tonic-gateperform the synchronization. While this is a severe limitation, only 128*0Sstevel@tonic-gateSUNWfiles currently uses this feature, and even that is only for backward 129*0Sstevel@tonic-gatecompatibility. We discourage use of this feature in future datastore 130*0Sstevel@tonic-gateimplementations. 131*0Sstevel@tonic-gate 132*0Sstevel@tonic-gateHow does it work? 133*0Sstevel@tonic-gate================= 134*0Sstevel@tonic-gate 135*0Sstevel@tonic-gateIt is helpful but not necessary to understand how this architecture works. 136*0Sstevel@tonic-gateFurthermore, the internal details are still evolving; if you rely on any 137*0Sstevel@tonic-gatedetails here, the only guarantee is that your code will break someday. 138*0Sstevel@tonic-gate 139*0Sstevel@tonic-gateThe easiest way to explain the architecture is by example; thus, assume you 140*0Sstevel@tonic-gatehave a module `mymod' that you want to use with DHCP Service Library 141*0Sstevel@tonic-gateSynchronization. Then, for each method specified in the DHCP Server 142*0Sstevel@tonic-gatePerformance Project specification, the following happens: 143*0Sstevel@tonic-gate 144*0Sstevel@tonic-gate 1. The private layer is called with the specified method 145*0Sstevel@tonic-gate (as specified in the DHCP Server Performance Project spec) 146*0Sstevel@tonic-gate 147*0Sstevel@tonic-gate 2. The private layer locates the underlying public module 148*0Sstevel@tonic-gate to invoke, given the settings in /etc/inet/dhcpsvc.conf. 149*0Sstevel@tonic-gate (as specified in the DHCP Server Performance Project spec) 150*0Sstevel@tonic-gate 151*0Sstevel@tonic-gate 3. The private layer detects that this module is one that 152*0Sstevel@tonic-gate requires use of DHCP Service Library Synchronization (by 153*0Sstevel@tonic-gate checking the value of the module's dsvc_synchtype symbol). 154*0Sstevel@tonic-gate 155*0Sstevel@tonic-gate 4. If this method is one for which synchronization guarantees 156*0Sstevel@tonic-gate are provided, the private layer sends a "lock" request 157*0Sstevel@tonic-gate across a door to the DHCP service door server daemon (also 158*0Sstevel@tonic-gate known as the lock manager), dsvclockd. 159*0Sstevel@tonic-gate 160*0Sstevel@tonic-gate 5. The dsvclockd daemon receives the lock request and attempts 161*0Sstevel@tonic-gate to lock a given container for either exclusive or shared 162*0Sstevel@tonic-gate access (depending on the request). If the lock request was 163*0Sstevel@tonic-gate "nonblocking" and the lock cannot be immediately acquired, 164*0Sstevel@tonic-gate a DSVC_BUSY error is returned. Otherwise, the daemon waits 165*0Sstevel@tonic-gate until it acquires the lock and sends a DSVC_SUCCESS reply 166*0Sstevel@tonic-gate back. 167*0Sstevel@tonic-gate 168*0Sstevel@tonic-gate 6. Assuming the lock could be obtained (if it was necessary; 169*0Sstevel@tonic-gate see step 4), the private layer locates the appropriate 170*0Sstevel@tonic-gate method in `ds_mymod.so' module, and calls it. 171*0Sstevel@tonic-gate 172*0Sstevel@tonic-gate 7. Once the method has completed (successfully or otherwise), 173*0Sstevel@tonic-gate if this was a method which required a "lock" request, the 174*0Sstevel@tonic-gate private layer sends an "unlock" request to the dsvclockd. 175*0Sstevel@tonic-gate 176*0Sstevel@tonic-gate 8. The private layer returns the reply to the caller. 177