Aubrey Barnard and Perry Kivolowitz

Our aim is 1) to write an extensible framework for measuring network performance over multiple adapters and 2) to develop techniques to analyze the captured data to suggest changes to how the adapters are being used.

Full project proposal: [to be included once I get permissions to upload external files]

These are my current notes on network metrics. These will be reformatted once I get a separate page for them.

From class and our brains:
jitter
heartbeat
DNS
DHCP
ping
simulate different traffic types, e.g. streaming multimedia
frequency of dropped packets
amount of unacked data
network saturation (achievable bandwidth)


----------
Google search

----------
Network Metric Report
http://www.geant2.net/upload/pdf/GN2-05-265v4-Deliverable_DJ1-2-3_Network_Metric_Report.pdf

Measurement dimensions:
Performance:
* Availability
* Loss and errors
* Delay
* Bandwidth
Miscellaneous:
* Device-specific
* Flow
* Routing

SNMP

measure bit errors with communicating synchronized PRNGs

measure bit errors at physical and data link layers

one-way delay, jitter, round trip time

bandwidth: capacity, utilization, available bandwidth, achievable bandwidth

capacity: use snmp (hardware endpoints should know capacity)

measure delay with synchronized clocks; problematic


----------
Non-metric coordinates for predicting network proximity
http://research.microsoft.com/pubs/73318/coords_infocom2008.pdf

Doesn't look very helpful as I'm not sure how min-plus applies to our situation. Also, overall it is an optimization procedure (iterative over time) which really doesn't help us.


----------
Loss Network Models and Multiple Metric Performance Sensitivity Analysis for Mobile Wireless Multi-hop Networks
http://www.isr.umd.edu/~vahidt/LossNetworkModel_WICON08.pdf

TODO


----------
Inference and Labeling of Metric-Induced Network Topologies
http://www.cs.bu.edu/fac/byers/pubs/tpds.pdf

hop-count

bottleneck bandwidth

packet loss rate

estimation of shared loss

TODO


----------
Controlling Multimedia QoS in the Future Home Network Using the PSQA Metric
http://comjnl.oxfordjournals.org/cgi/reprint/49/2/137

TODO


----------
Improving network anomaly detection effectiveness via an integrated multi-metric-multi-link PCA-based approach
http://www3.interscience.wiley.com/cgi-bin/fulltext/121481651/PDFSTART

TODO


----------
IMC '08

----------
A measurement study of a commercial-grade urban wifi mesh
http://portal.acm.org/citation.cfm?id=1452520.1452534&coll=portal&dl=ACM&type=series&idx=SERIES10693&part=series&WantType=Proceedings&title=IMC&CFID=26429339&CFTOKEN=74708467

SNMP

number failed transmissions, wireless noise floor

iperf (TODO)


----------
IMC '07

----------
Characterizing residential broadband networks 
http://portal.acm.org/citation.cfm?id=1298306.1298313&coll=portal&dl=ACM&type=series&idx=SERIES10693&part=series&WantType=Proceedings&title=IMC&CFID=26429339&CFTOKEN=74708467

queue lengths, seems residential ISPs have large traffic queues which delay packets by milliseconds

do we account for upstream vs. downstream differences?

1488 byte TCP ACK packets

floods, trickles


----------
Studying wireless routing link metric dynamics
http://portal.acm.org/citation.cfm?id=1298306.1298352&coll=portal&dl=ACM&type=series&idx=SERIES10693&part=series&WantType=Proceedings&title=IMC&CFID=26429339&CFTOKEN=74708467

expected transmission count: how many transmissions do you expect to need to successfully transmit a packet
1/(P(packet success)*P(ack success))

measure bandwidth via packet pair, back-to-back probe packets of increasing size


----------
IMC '04

----------
Self-configuring network traffic generation
http://portal.acm.org/citation.cfm?id=1028788.1028798&coll=portal&dl=ACM&type=series&idx=SERIES10693&part=series&WantType=Proceedings&title=IMC&CFID=26429339&CFTOKEN=74708467

Related work, nothing helpful on metrics


----------
Strategies for sound internet measurement
http://portal.acm.org/citation.cfm?id=1028788.1028824&coll=portal&dl=ACM&type=series&idx=SERIES10693&part=series&WantType=Proceedings&title=IMC&CFID=26429339&CFTOKEN=74708467

precision: what data is kept/omitted; timing;

justify your measurement decisions

associate meta-data with measurements

accuracy: e.g. what can go wrong with packet filtering, several levels can drop; stable clock rate

misconception: are you measuring what you think you are measuring?; beware of proxies; large socket buffers and small transfer sizes; vantage point; representativeness (i.e. controlled experiments)

compute connection size by difference in TCP SYN/FIN/RST sequence numbers

calibration: outliers, spikes often suggest measurement problems; self-consistency checks; multiple ways of measuring same thing; test using synthetic data


----------
Ten fallacies and pitfalls on end-to-end available bandwidth estimation
http://portal.acm.org/citation.cfm?id=1028788.1028825&coll=portal&dl=ACM&type=series&idx=SERIES10693&part=series&WantType=Proceedings&title=IMC&CFID=26429339&CFTOKEN=74708467

available bandwidth metric is link capacity above and beyond average utilization

mind the relation between probing stream duration and averaging time scale

packet trains are better than packet pairs

beware cross-traffic burstiness

beware multiple bottlenecks

beware iterative probing does not necessarily converge to a single number

available bandwidth should not be compared to bulk TCP throughput


----------
Single-hop probing asymptotics in available bandwidth estimation: sample-path analysis
http://portal.acm.org/citation.cfm?id=1028788.1028831&coll=portal&dl=ACM&type=series&idx=SERIES10693&part=series&WantType=Proceedings&title=IMC&CFID=26429339&CFTOKEN=74708467

explains how to analyze packet trains


----------
Bandwidth estimation in broadband access networks
http://portal.acm.org/citation.cfm?id=1028788.1028832&coll=portal&dl=ACM&type=series&idx=SERIES10693&part=series&WantType=Proceedings&title=IMC&CFID=26429339&CFTOKEN=74708467

nothing significant


----------
SIGCOMM '91

----------
A control-theoretic approach to flow control
http://portal.acm.org/citation.cfm?id=115992.115995&coll=portal&dl=ACM&type=series&idx=SERIES419&part=series&WantType=Proceedings&title=COMM&CFID=26429339&CFTOKEN=74708467

packet-pair technique: measure ack delay to estimate server's processing rate

Proposal submitted.

Meeting to address project feasibility, delegate tasks, discuss design. AB will work on mining metrics from literature. PK will work on developing client. To meet again on Thursday.

AB - Learned about TCP so I can figure out some ways to measure it.

PK - Got enumeration of adapters running along with capture on selected adapters. Simple GUI to enable / disable adapter capture.

Met briefly - discussed evolving approach. Will meet again on 03/06.

PK - Learned how to create dynamically loaded plug-in interface so that metrics / tests can be self-standing modules.

Met briefly - discussed metrics as known so far. Will meet again Monday.

PK - Wrote a DNSQuery plug-in using low level socket programming so that the query can be sent over non-default interfaces. Plug-in works over default interface (confirmed with WireShark) - have to add another NIC to my computer to test the non-default case.

PK - Added second adapter to desktop. Confirmed plugins are working over alternate adapters. Wrote PING plugin using low level socket interface. This is working nicely. Started traffic driver UI.

AB - Considered reorganizing web pages. Learned to upload files, but held uploading proposal until after reorg. due to possible linking/versioning problems. Researched network metrics from Google search results. Started finding papers from Sigmetrics and related conferences (IMC, MobiCom).

Notes:

• Measure bit error rates (physical and link layers should be able to report)
• Get capacity of links
• Measure achievable bandwidth by saturation

AB, PK - Meeting to discuss what has been learned about metrics so far, demonstrate current client and server functionality. Meet again on Thursday (03/12). AB to complete literature search, learn about SNMP. PK to continue work on client/server.

PK - Server is beginning to take shape. Have architecture mapped out to permit user level and packet level processing to take place in a single application. Server and client have hooked up both within my LAN and from outside to inside my LAN. Server is completely multithreaded and has already handled multiple concurrent clients.

PK - TCPPlugin and server are now happily exchanging traffic.

AB - Went through abstracts of last two years of SIGMETRICS IMC.

AB - Finished literature search of SIGMETRICS IMC. At this point I think I've collected about enough information from the literature. Read about SNMP.

AB - Creating new site organization. The new site should begin here: Multi-Interface Traffic Scheduling. Added temporary version of metrics notes.

PK - After much coding, the traffic driver is fully running including 3 different types of traffic and real time capture at the hardware level. Next step is to add real time capture to the server. Not including experiments, about 2200 lines of code have been written so far.

AB - Worked some on project wiki site. Insufficient permissions yet to reorganize and do what I want.

PK - Created UDP plug-in. Server is sinking them. Next step is still real time capture on the server.