btdht.utils module

bencode bencode an arbitrary object
bdecode bdecode an bytes string
bdecode_rest bdecode an bytes string
enumerate_ids Enumerate 2 to the power of size ids from id
id_to_longid convert a random bytes to a unicode string of 1 and 0
ip_in_nets Test if ip is in one of the networks of nets
nbit Allow to retrieve the value of the nth bit of s
nflip Allow to flip the nth bit of s
nset Allow to set the value of the nth bit of s
ID A 160 bit (20 Bytes) string implementing the XOR distance
PollableQueue A queue that can be watch using select.select()
Scheduler Schedule weightless threads and DHTs io
btdht.utils.bencode(obj)

bencode an arbitrary object

Parameters:obj – A combination of dict, list, bytes or int
Returns:Its bencoded representation
Return type:bytes
Notes:
This method is just a wrapper around _bencode()
btdht.utils.bdecode(s)

bdecode an bytes string

Parameters:s – A bencoded bytes string
Returns:Its bencoded representation
Return type:A combination of dict, list, bytes or int
Raises:BcodeError – If failing to decode s
Notes:
This method is just a wrapper around _bdecode()
btdht.utils.bdecode_rest(s)

bdecode an bytes string

Parameters:s – A bencoded bytes string
Returns:A couple: (bdecoded representation, rest of the string). If only one bencoded object is given as argument, then the ‘rest of the string’ will be empty
Return type:tuple ( combination of dict, list, bytes or int, bytes)
Raises:BcodeError – If failing to decode s
btdht.utils.enumerate_ids(size, id)

Enumerate 2 to the power of size ids from id

Parameters:
  • size (int) – A number of bit to flip in id
  • id (bytes) – A 160 bit (20 Bytes) long id
Returns:

A list of id and 2 to the power of size (minus one) ids the furthest from each other

Return type:

list

For instance: if id=("\0" * 20) (~0 * 160), enumerate_ids(4, id) will return a list with

  • '\x00\x00\x00\x00\x00...' (~00000000…)
  • '\x80\x00\x00\x00\x00...' (~10000000…)
  • '@\x00\x00\x00\x00.......' (~0100000000…)
  • '\xc0\x00\x00\x00\x00...' (~11000000…)

The can be see as the tree:

       \x00
       /  \
     1/    \0
     /      \
   \xc0    \x00
  1/ \0    1/ \0
  /   \    /   \
\xc0 \x80 @   \x00

The root is id, at each level n, we set the nth bit to 1 left and 0 right, size if the level we return.

This function may be usefull to lanch multiple DHT instance with ids the most distributed on the 160 bit space.

btdht.utils.id_to_longid(id, l=20)

convert a random bytes to a unicode string of 1 and 0

For instance: "\0" -> "00000000"

Parameters:
  • id (bytes) – A random string
  • size (int) – The length of id
Returns:

The corresponding base 2 unicode string

Return type:

unicode

btdht.utils.ip_in_nets(ip, nets)

Test if ip is in one of the networks of nets

Parameters:
  • ip (str) – An ip, in dotted notation
  • nets (list) – A list of netaddr.IPNetwork
Returns:

True if ip is in one of the listed networks, False otherwise

Return type:

bool

btdht.utils.nbit(s, n)

Allow to retrieve the value of the nth bit of s

Parameters:
  • s (bytes) – A byte string
  • n (int) – A bit number (n must be smaller than 8 times the length of s)
Returns:

The value of the nth bit of s (0 or 1)

Return type:

int

btdht.utils.nflip(s, n)

Allow to flip the nth bit of s

Parameters:
  • s (bytes) – A byte string
  • n (int) – A bit number (n must be smaller than 8 times the length of s)
Returns:

The same string except for the nth bit was flip

Return type:

bytes

btdht.utils.nset(s, n, i)

Allow to set the value of the nth bit of s

Parameters:
  • s (bytes) – A byte string
  • n (int) – A bit number (n must be smaller than 8 times the length of s)
  • i (int) – A bit value (0 or 1)
Returns:

s where the nth bit was set to i

Return type:

bytes

class btdht.utils.ID

Bases: object

A 160 bit (20 Bytes) string implementing the XOR distance

Parameters:id – An optional initial value (bytes or ID). If not specified, a random 160 bit value is generated.
value = None

bytes, Actual value of the ID

classmethod to_bytes(id)
Parameters:id – A bytes or ID
Returns:The value of the id
Return type:bytes
startswith(s)

S.startswith(prefix[, start[, end]]) -> bool

Return True if S starts with the specified prefix, False otherwise. With optional start, test S beginning at that position. With optional end, stop comparing S at that position. prefix can also be a tuple of strings to try.

__getitem__(i)

x.__getitem__(y) <==> x[y]

__xor__(other)

Perform a XOR bit by bit between the current id and other

Parameters:other – A bytes or ID
Returns:The resulted XORed bit by bit string
Return type:bytes
class btdht.utils.PollableQueue

Bases: Queue.Queue

A queue that can be watch using select.select()

Parameters:maxsize (int) – The maximum size on the queue. If maxsize is <= 0, the queue size is infinite.
sock = None

A socket.socket object ready for read then here is something to pull from the queue

empty()

Return True if the queue is empty, False otherwise (not reliable!).

full()

Return True if the queue is full, False otherwise (not reliable!).

get(block=True, timeout=None)

Remove and return an item from the queue.

If optional args ‘block’ is true and ‘timeout’ is None (the default), block if necessary until an item is available. If ‘timeout’ is a non-negative number, it blocks at most ‘timeout’ seconds and raises the Empty exception if no item was available within that time. Otherwise (‘block’ is false), return an item if one is immediately available, else raise the Empty exception (‘timeout’ is ignored in that case).

get_nowait()

Remove and return an item from the queue without blocking.

Only get an item if one is immediately available. Otherwise raise the Empty exception.

join()

Blocks until all items in the Queue have been gotten and processed.

The count of unfinished tasks goes up whenever an item is added to the queue. The count goes down whenever a consumer thread calls task_done() to indicate the item was retrieved and all work on it is complete.

When the count of unfinished tasks drops to zero, join() unblocks.

put(item, block=True, timeout=None)

Put an item into the queue.

If optional args ‘block’ is true and ‘timeout’ is None (the default), block if necessary until a free slot is available. If ‘timeout’ is a non-negative number, it blocks at most ‘timeout’ seconds and raises the Full exception if no free slot was available within that time. Otherwise (‘block’ is false), put an item on the queue if a free slot is immediately available, else raise the Full exception (‘timeout’ is ignored in that case).

put_nowait(item)

Put an item into the queue without blocking.

Only enqueue the item if a free slot is immediately available. Otherwise raise the Full exception.

qsize()

Return the approximate size of the queue (not reliable!).

task_done()

Indicate that a formerly enqueued task is complete.

Used by Queue consumer threads. For each get() used to fetch a task, a subsequent call to task_done() tells the queue that the processing on the task is complete.

If a join() is currently blocking, it will resume when all items have been processed (meaning that a task_done() call was received for every item that had been put() into the queue).

Raises a ValueError if called more times than there were items placed in the queue.

class btdht.utils.Scheduler

Bases: object

Schedule weightless threads and DHTs io

A weightless threads is a python callable returning an iterator that behave as describe next. The first returned value must be an integer describing the type of the iterator. 0 means time based and all subsequent yield must return the next timestamp at which the iterator want to be called. 1 means queue based. The next call to the iterator must return an instance of PollableQueue. All subsequent yield value are then ignored. The queue based iterator will be called when something is put on its queue.

zombie
Returns:True if the scheduler is stoped but its threads are still running
Return type:bool
start(name_prefix="scheduler")

start the scheduler

Parameters:name_prefix (str) – Prefix to the scheduler threads names
stop()

stop the scheduler

Raises:FailToStop – if we fail to stop one of the scheduler threads after 30 seconds
stop_bg()

Lauch the stop process of the dht and return immediately

is_alive()

Test if the scheduler main thread is alive

Returns:True the scheduler main thread is alive, False otherwise
Return type:bool
thread_alive(name)

Test is a weightless threads named name is currently schedule

Parameters:name (str) – The name of a thread
Returns:True if a thread of name name if found
Return type:bool
add_dht(dht)

Add a dht instance to be schedule by the scheduler

Parameters:dht (dht.DHT_BASE) – A dht instance
del_dht(dht)

Remove a dht instance from the scheduler

Parameters:dht (dht.DHT_BASE) – A dht instance
add_thread(name, function, user=False)

Schedule the call of weightless threads

Parameters:
  • name (str) – The name of the thread to add. Must be unique in the Scheduler instance
  • function – A weightless threads, i.e a callable returning an iterator
  • user (bool) – If True the weightless threads is schedule in a secondary thread. The default is False and the weightless threads is processed in the main scheduler thread. This is usefull to put controled weightless threads and the main thread, and all the other (like the user defined on_``msg``_(query|response)) function to the secondary one.
del_thread(name, stop_if_empty=True)

Remove the weightless threads named name

Parameters:
  • name (str) – The name of a thread
  • stop_if_empty (bool) – If True (the default) and the scheduler has nothing to schedules, the scheduler will be stopped.