Lists

The basic List interface resembles an array, in that one can access and update elements by specifying their position. The underlying implementation, however, may not be via an array (for example, linked lists), in which case accessing the Nth element may not be as fast as accessing a corresponding array element.

Most list interfaces also support some mechanism of automatic size expansion to accommodate new additions, reducing the risk of array-bounds overflow. However, attempting to access an element (by position) that is not there is always an error: retrieving item 100 of a list that contains items 0 through 37 can never end well.

Java List<T> and ArrayList<T>

Java supports the ArrayList class, and the List interface. We'll only touch on interfaces here.

To declare an ArrayList of strings, use

    import java.util.*;    // or java.util.ArrayList
    ...
    ArrayList<String> L = new ArrayList<String>()

The constructor here is the right side; there is no parameter. The <String> is the "type parameter". ArrayList is a generic class, as are most data-structure classes.

The list L created above is initially empty. We use L.add(str) to add entries:

    L.add("apple");
    L.add("banana");
    L.add("cherry");

The current size of the list is available as L.size(); this would return 3 for the list above. The elements are at positions 0, 1 and 2.

We can now set and retrieve elements with L.set(i,val) and L.get(i), corresponding to A[i]=val and A[i], for an array A. In the above, L.get(0) returns "apple".

• L.contains(val)         returns true if val occurs in L
• L.indexOf(val)          finds position of first occurrence of val in L, or -1 if val is not there
• L.remove(n)             removes value at position n
• L.remove(val)          finds first occurrence of val and removes it
• L.subList(from, to)  returns new list of L[from],L[from+1], ..., L[to-1]
• L.toArray()              returns an array, with no excess capacity

Vector

Chapter 3 contains Bailey's Vector class, which represents a List of Objects implemented via an array. The main feature is that the vector can grow.

We'll start with the Vector interface on page 46. 

Bailey's examples:

    Vector used for Wordlist

    Vector used for L-systems

There is an issue with Vectors of Objects: we don't really want Objects. Look closely at the code on page 47; it contains a cast to (String):

    targetWord = (String) list.get(index);

What this does is takes the Object list.get(index), and, because it is really a string, allows its use as a string at compile-time.

In Java we will usually use generics, eg Vector<T>.

In the word-frequency example on page 49 (actually starting on p 48) there is

    wordInfo = (Association) vocab.get(i);

and

    vocabWord = (String) wordInfo.getKey();

Adding to the middle of a vector: you need to move from right to left. See the picture on Bailey p 52.

Vector Growth Demo

C# version

Java version

• addOne(String s): to add one more string
• addSome(int count, String s): to add multiple copies of the string
• addArray(int count, String s): this works like addSome(count,s), but it puts all the new strings into a new ArrayList, and then adds them all at once to the original list.

Try:

1. Use the inspector to examine the addOne("apple"). Inspect theList. What is the size of elementData? Where is elementData, by the way? Where did it come from?
2. Same after addOne("apple")
3. Same after addSome(9,"banana"). Why 9?
4. Same after addOne("cherimoya")

Keep going until you can guess the expansion pattern for elementData.

§3.5: analysis of costs of expansion

• The number of moves to insert at a random place in the middle of a list of length N is, on average, N/2.
• The number of compares to search a list of length N for a random element that is in fact present is, on average, N/2.
• The number of compares to search a list of length N for an element not there is simply N.

Search and insert costs

Generics

We would really like to be able to declare containers with a fixed type, where the type is supplied as parameter:

    List<String> s = new List<String>();

If we use Bailey's Vector class, we would have pretty much the same performance, but getting strings out of the Vector would always require a cast:

    String s = (String) vect.get(3);

Stack

• s.push(A): adds data value A to the stack
• s.pop(): returns the most recently pushed value, and deletes it from the stack

Implementing a stack

Deleting from a list

If all we do is add, then the growth strategy of doubling the internal space when necessary makes perfect sense.

But what happens if we will regularly grow lists to large size, and then delete most of the entries? A list grown to have internal capacity 1024 will retain that forever, even if we shrink down to just a few elements.

One approach is to re-allocate to a smaller elements[] whenever L.Count < L.Capacity/2, or something like that.

Morin in §2.6 (p 49) introduces what he calls a RootishArrayStack, which is an array-based list with an efficient delete operation. Here are the key facts (big-O notation is officially introduced in the next section):

• The space used for n elements is n + O(√n)
• For any m add/remove operations, the time spent growing and shrinking is O(m)

The idea is to keep a list of arrays (an array of pointers to arrays). These sub-arrays have size 1, 2, 3, 4, etc respectively. For 10 elements, the RootishArrayStack would have four arrays, and thus a capacity of 1+2+3+4.

If the RootishArrayStack has N elements in n arrays, then N ≃ n²/2; this follows because 1+2+...+n ≃ n²/2. When the RootishArrayStack needs to expand, it will add an array of size n+1 to the pool; this is about √(2n). Thus, growth is "slower" than for C# Lists or Java ArrayLists. However, when a new allocation is made for growth, the old space is not discarded.

The real advantage of the RootishArrayStack is for deletions. If the list shrinks so that the last sub-array is now empty, that sub-array and that sub-array only is discarded. This is a relatively efficient operation.

Big-O notation and Bailey Chapter 5: Analysis

A function is said to be polynomial if it is O(n^k) for some fixed k; quadratic growth is a special case.

So far we've been looking mainly at running time. We can also consider space needs. As an example, see the Table of Factors example on Bailey page 88. Let us construct a table of all the k<=n and a list of all the factors (prime or not) of k, and ask how much space is needed. This turns out to be n log n. The argument here is a bit mathematical; see Bailey. If the table length is n, then factor f can appear no more than n/f times (once every fth line).

The running time to construct the table varies with how clever the algorithm is, it can be

• O(n²) [check all i<k for divisibility]
• O(n^3/2) [check all i<sqrt(k)]
• O(n log n) [Sieve of Eratosthenes]

Now suppose we want to search a large string for a specific character. How long should this take? Bailey has an example on p 90. The answer depends on whether we're concerned with the worst case or the average case (we are almost never interested in the best case). If the average case, then the answer typically depends on the probability distribution of the data.

Linked Lists

• Bailey, chapter 9 section 4
• Morin, Chapter 3 (p 63) (we will look at singly-linked lists, or Morin's SLList)

    public class Cell<T> {
       private T data_;
       private Cell<T> next_;
       public Cell(T s, Cell<T> n) {data_ = s; next_ = n;}
       public T data() {return data_;}
       public Cell<T> next() {return next_;}
       public void setData(T s) {data_ = s;}
       public void setNext(Cell<T> c)   {next_ = c;}
    }

Implementing a stack using linkedlist

Implementing a set

Java LinkedList

Using List<T> to implement a Matrix class in C#

• Matrix(int height, int width)
• getWidth()
• getHeight()
• get(int row, int col)
• set(int row, int col, double val)

How should we proceed?

Here's a simpler problem: how should we implement a Vector<T> class, where vector objects have a fixed length, and are initialized to 0? C# does take care of that latter, but List<T>'s do not automatically have the right length. Also, ideally we'd like to "hide" the add() operation, that can make a List<T> grow longer than we'd like.

Now let's return to the Matrix class. As in Vector, we will pre-allocate space for all the elements. Here is some simple code to implement a matrix class with TList objects (not yet converted to Java).

/**
 * Class Matrix is implemented by a TList of rows.
 */

class Matrix {
    // instance variables - replace the example below with your own
    private TList<TList<double> > m;
		// list of lists
    private int height, width;

    /**
     * Constructor for objects of class TList
     */
    public Matrix(int h, int w)
    {
        // initialize instance variables
        height = h;
	width = w;
        m = new TList<TList<double>>(height);
	// we must preallocate all the rows
	for (int i = 0; i<height; i++) {
		TList<double> theRow = new TList<double>(width);
		theRow.Fill(0.0);
		// we must preallocate all the slots (columns) in each row
		m.Add(theRow);
	}
    }
       
    public int getwidth() {return width;}
    public int getheight() {return height;}

    // get nth value, with range check
    public double get(int r, int c) {
        if (r<0 || r >= height) {
            Console.WriteLine("Warning: Matrix.get() called with out-of-range row = " + r);
            return 0.0;	
        }
        if (c<0 || c >= width) {
            Console.WriteLine("Warning: Matrix.get() called with out-of-range column = " + c);
            return 0.0;	
        }
        return m.get(r).get(c);
    }
    
    // set nth value, with range check
    public void set(int r, int c, double val) {
        if (r<0 || r >= height) {
            Console.WriteLine("Warning: Matrix.set() called with out-of-range row = " + r);
            return;	
        }
        if (c<0 || c >= width) {
            Console.WriteLine("Warning: Matrix.get() called with out-of-range column = " + c);
            return;	
        }
        m.get(r).set(c,val);
    }    
}

• We're using TList to build a 2-D structure.
• There's no analogue to TList.add(E e); we have to add entire rows or columns or else the Matrix will no longer be neatly rectangular. Note that we add new rows and columns "empty", that is, populated with nulls. (In the code above, there is no way to add a new row or column.)
• Because the generic class uses TLists, not arrays, we don't have any problem using the element type E directly throughout. When we created the Vector and MyList classes, we had that annoying need to use Object when creating arrays even when we wanted EltType.
• Matrix.print(int fieldwidth) is a handy way of generating output. Note the parameter. Because of the parameter, making this into ToString() is tricky. (Not shown above.)
• How do we know all the rows are the same length?

Linked List Efficiency

Stacks and Linked Lists

While the array implementation of a stack is quite fast, the linked list approach is equally straightforward. All we have to do is maintain a pointer to the head:

class stack<T> {
    private Cell<T> head_;
    public bool is_empty() {return (head_ == null);}
    public T pop() {T val = head_.data(); head_ = head_.next(); return val;}
    public void push(T val) {head_ = new Cell<T>(val, head_);}
}

What would we need to do in C++ if we wanted to be sure we deleted a popped cell?

Sorting Linked Lists

How would you sort a linked list? QuickSort is out.

Hashing

• int values
• "d"
• "A"
• " "
• "2"

    "avocado",
    "banana",
    "canteloupe",
    "durian",
    "eggplant",
    "feijoa",

{
    "avocado",
    "banana",
    "canteloupe",
    "durian",
    "eggplant",
    "feijoa",
    "guava",
    "hackberry",
    "iceberg",
    "jicama",
    "kale",
    "lime",
    "mango",
    "nectarine",
    "orange",
    "persimmon",
    "quince",
    "rutabega",
    "spinach",
    "tangerine",
};

Open Hashing

Another way to do hashing is so-called "open" hashing: a data object d is simply put into htable[hash(d)]. If that position is taken, the next position is used. For this to work, we need to be sure that htablesize is quite a bit larger (eg at least double) the number of elements added. Deletions require careful thought. See Bailey 15.4.1.

Traversing a Hash Table

A class based on this is in hashtable.java; note the print() method. This class uses the string type; there is also inthashclass.cs that uses int (yes, I should have made this use a generic type).

Hash-table performance

The usual strategy is to choose a table size comparable to the number of items stored, rehashing as necessary to maintain this as the number of items grows. This way, the average length of the bucket lists is 1.

This isn't quite as good as it sounds, as the empty buckets figure in the length average but not into the real-world performance stats. Still, if λ is the average number of items per bucket, and N is the number of buckets, then the Poisson distribution says that the expected number of buckets with k items is Ne^-λ/k!. If λ=1, this means that list lengths are approximately distributed as follows:

The above assumes that the hash function distributes items among the buckets randomly. Trying to improve on this is usually not worth the effort. However, for special cases when there are many times more lookups than updates, it may pay to attempt to tweak the hash function to minimize collisions; this would have to be done after every few updates though. One common approach to tweaking is to try a range of different values for some numeric parameter built into the hash function, and then pick the value that makes the hash function perform best.

Fibonacci Hashing

The performance bottleneck for classic hashing is dividing by the size of the table. Here's an alternative: the table size is a power of 2, M, and we have, for integer x

    hash(x) = trunc(M×(a×x mod W)/W)

(For string data, we might use hash(s.hashCode().) W is the word size of the machine, eg 2³². M is the table size, also a power of 2 (perhaps M = 2¹⁰ = 1024). The value a is W/φ, where φ is the Fibonacci ratio, or the golden ratio, (√5+1)/2. For W=32, a=2,654,435,769.

The reason this often works well is that it is particularly effective at spreading the hash values of consecutive x's very widely. When used with open hashing, above, this means we will seldom encounter collisions. Multiplication by the large value of a, above, is usually quite a bit faster than dividing by a non-power-of-two table size. Finding a×x modulo the word size W just means that we did the multiplication and got the low-order bits only; that is, we ignored the overflow.

Hash Sets and Hash Dictionaries

class StrSet {

    private StrList sl;
    public StrSet() {sl = new StrList(100); }
    public boolean isMember(String s) {
        for (int i=0; i<sl.size(); i++) {
            if (sl.get(i) == s) return true;
        }
        return false;
    }

    public void add(string s) {
	if (isMember(s)) return;
	sl.add(s);
    }
}
      

class hashset {
    private hashtable ht;
    public hashset(int size) {ht = new hashtable(size);}

    public boolean isMember(String s) {
        return ht.isMember(s);
    }

    public void add(String s) {
	if (isMember(s)) return;
	ht.add(s);
    }

    public void print() {
	ht.print();
    }
}

Dictionaries

• V get (K key): returns the value corresponding to key, or else default(V) (generally null)
• void add(K key, V val): adds the new pair. Precondition: K is not already present
• void update(K key, V newval): like add. K may or may not be present.

• Use s.Split(" .,;[]()\t") to split at other characters besides spaces
• Convert each word to lowercase

Iterators

Suppose we build a hashtable object, ht, of Strings. Suppose we want to be able to print out the hash table using a for-each loop, like this:

foreach (String s: ht) System.out.println(s);

What do we have to do? The answer is to define an iterator. If we just include the foreach loop above, we get this error message:

Error: java: for-each not applicable to expression type
  required: array or java.lang.Iterable
  found:    hashtable

We must make hashtable implement the Iterable interface, as follows:

class hashtable implements Iterable<String> {

To implement an interface is to promise to implement within the class the methods required by that interface. In this case, we must have a method of type public Iterator<String> iterator(). To do this, in turn, we typically have it return an iterator object that we define for our class:

   public Iterator<String> iterator() {
       return new hIterator();
   }

Now we have to define class hIterator, which must implement Iterator<String> (not the same as Iterable<String> above!). An iterator must implement the following methods:

• boolean hasNext();
• String next();
• void remove();

The first two are all we need for the system to be able to iterate through the table, returning each successive entry with each call to next:

while (hasNext()) {
    System.out.println(next());
}

The iterator keeps track of our position. The idea is that the internal state of the iterator should always refer to the next element in the structure.

Before the HashTable example, we'll start with something simpler: an iterator for an ArrayList class [arraylistiterator]. The variable pos  in the iterator class below keeps track of the position of the next element of the array; it is initialized to 0 and hasNext() becomes false when pos == elements.length. We don't implement remove().

private class alIterator implements Iterator<String> {
    private int pos;

    public alIterator() {
        System.out.println("initializing alIterator");
        pos = 0;
    }

    public boolean hasNext() {
        return (pos < elements.length);
    }

    public String next() {
        String retval = elements[pos];
        pos++;
        return retval;
    }

    public void remove() {
        throw new UnsupportedOperationException();
    }
}

Now let's do this for a hashtable. We'll need two variables to keep track of where we are, row and p; p will be a pointer to Cell somewhere in the list htable[row]. We'll advance to a non-null p initially. After returning p.getString(), we'll advance to the next non-null p; in most cases this should be p.next() but sometimes we'll have to advance one or more rows as well. The private method findnext() takes care of advancing to further rows in the event that p==null.

private class hIterator implements Iterator<String> {
   private int row;
   private Cell p;

   public hIterator() {
      System.out.println("initializing hIterator");
      row = 0;
      p = htable[row];
      findnext();
   }

   public boolean hasNext() { return (p != null && row < htablesize); }

   public String next() {
      String retval = p.getString();
      p = p.next();
      findnext();
      return retval;
   }

   private void findnext() {
      while (p==null) {
         row++;
         if (row >= htablesize) break;
         p = htable[row];
      }
   }

   public void remove() { throw new UnsupportedOperationException(); }
}

	foreach (KeyValuePair<string,int> kvp in d) 
		Console.WriteLine("{0}: {1}", kvp.Key, kvp.Value);

    public class Cell {
	private K key_;
	private V val_;
	private Cell  next_;
	public Cell(K k, V v, Cell n) {key_ = k; val_ = v; next_ = n;}
	public K getKey() {return key_;}
	public V getVal() {return val_;}
	public Cell   next() {return next_;}
        public void setVal(V v) {val_ = v;}
        public void setNext(Cell c)   {next_ = c;}
    }	

	class dictionary : System.Collections.Generic.IEnumerable> {

    IEnumerator> IEnumerable>.GetEnumerator() {
	return foonumerator();
    }

    IEnumerator> foonumerator() {
	for (int i = 0; i(p.getKey(), p.getVal());
		p = p.next();
	    }
	}
	yield break;
    }

    System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator() { 
	return foonumerator(); 
    }